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Abstract 

This  research  focused  on  the  use  of  faces  as  a  means  of  representing 
contractor  generated  cost  performance  report  (CPR)  data.  The  study- 
investigated  the  feasibility  of  using  faces,  and  whether  the  use  of 
faces  would  lead  to  effective  decision  making  based  on  the  information 
that  the  faces  represent.  The  faces  concept  involves  the  transformation 
of  data  into  cartoon-like  faces  by  pairing  specific  variables  of  the 
data  set  with  distinctive  facial  features,  such  as  pupil  location  and 
nose  length. 

Feasibility  of  use  was  defined  in  terms  of  the  availability  of  a 
personal  computer  based  program  that  could  be  used  to  draw  the  faces  and 
whether  individuals  would  feel  confident  using  the  faces.  Effective 
decision  making  is  defined  as  the  ability  to  make  accurate  decisions  in 
a  reasonable  period  of  time. 

A  computer  program  was  developed  to  draw  the  faces.  A  test  scenario 
was  designed  to  evaluate  a  user's  decision  making  confidence  and 
decision  making  effectiveness.  The  test  utilized  the  tabular  data 
presentation  mode  as  a  basis  for  comparison. 

A  statistical  analysis  of  the  data  collected  indicated  that  the  use 
of  faces  resulted  in  lower  decision  making  confidence  and  lower  decision 
making  effectiveness  as  compared  to  the  use  of  tables.  However, 
training  and  experience  in  the  use  of  faces  significantly  increases  both 
the  confidence  and  effectiveness  levels. 
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AN  INVESTIGATION  INTO  THE  USE  OF  FACES  AS  A 
MEANS  OF  PRESENTING  COST  PERFORMANCE  REPORT  DATA 


I .  Introduction 


Genera 1  Issue 

The  method  of  using  cartoon  faces  to  represent  information  (facial 
representation)  may  be  useful  in  the  analysis  of  Air  Force  contractor 
generated  cost  performance  report  (CPR)  data.  The  use  of  cartoon  faces 
to  represent  information  was  developed  in  1973.  The  method  employs  a 
technique  of  matching  certain  facial  characteristics  with  specific 
variables  of  the  information  set.  The  underlying  strength  of  this 
method  is  that  as  the  value  of  the  variable  changes,  so  does  the  shape 
of  the  facial  characteristic.  This  allows  the  investigator  to  recognize 
relationships  between  the  variables  of  the  information  set.  The  faces 
could  be  drawn  by  hand  or  by  computer.  Ebcperiments  have  been  done  to 
determine  whether  this  method  would  be  beneficial  in  analyzing  certain 
types  of  financial  information.  The  results  suggest  that  a  beneficial 
potential  does  exist;  however,  additional  research  is  recommended .  To 
date,  the  Air  Force  has  not  accomplished  any  research  in  this  area. 

Specific  Problem 

The  objective  of  this  research  is  to  determine  if  faces  can  be  used 
for  presenting  and  analyzing  Air  Force  contractor  generated  cost 
performance  data,  and  to  determine  if  these  faces  can  be  used  as  an 
effective  decision  making  tool. 
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Scope  and  Limitations 


This  study  will  compare  the  effectiveness  of  tabular  and  facial 
presentation  in  assisting  a  program  manager  in  his/her  evaluation  of 
contractor  cost  performance  data.  Because  no  software  previously 
existed  to  draw  faces  representing  such  data,  this  research  has  included 
the  development  of  such  software.  After  the  software  was  developed,  it 
was  used  to  prepare  a  series  of  faces  for  comparison  with  tabular  data. 

Test  subjects  used  in  conducting  the  comparison  experiment  were 
limited  to  those  individuals  currently  attending  the  Air  Force  of 
Technology  (AFIT)  in  residence,  specifically  those  in  the  Graduate 
Systems  Management  (GSM) ,  Graduate  Contracting  Management  (GCM) ,  and 
Graduate  Cost  Analysis  (GCA)  Programs.  A  knowledge  of  cost  performance 
measurement  was  not  necessary,  however,  the  population  was  limited  to 
those  who.  because  of  their  career  fields,  were  familiar  with  its 
concept  and  will  most  likely  be  requiring  cost  performance  measurement 
tools  in  the  future. 

The  primary  emphasis  of  the  experiment  is  decision  making 
effectiveness.  Based  on  previous  research  in  this  area  and  in  the  area 
of  graphical  presentations  in  general,  the  following  are  considered  to 
be  the  major  variables  of  interest  (5:4;  11:207): 

Dependent  Variables  -  dec  3ion  accuracy  and  speed;  and 

Independent  Variables  -  user  background  and  mode  of  presentation. 
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Investigative  Questions 


The  following  questions  were  used  to  direct  the  focus  of  this 
research : 

1)  How  can  the  method  of  facial  representation  be  personal  computer 
(PC)  based?  The  answer  to  this  question  required  an  actual  PC  based 
software  package. 

2)  An  essential  aspect  of  facial  representation  is  the  matching  of 
variables  to  facial  characteristics.  It  is  necessary  to  determine  what 
matches  would  be  appropriate  in  the  realm  of  Air  Force  contractor 
generated  cost  and  performance  data.  Two  questions  that  should  be 
answered  are: 

a)  What  variables  should  be  selected? 

b)  Which  facial  characteristics  should  be  tied  to  each 

variable? 

3)  How  effective  is  facial  representation  in  conveying  cost 
performance  measurement  data?  For  example,  how  useful  is  it  as  a 
decision  making  tool  in  decisions  about  contractor  performance? 

4)  How  does  a  person's  level  of  experience  impact  the  usefulness  of 
Lie  faces  method? 

The  remainder  of  this  study  will  be  devoted  to  answering  those  four 
questions.  Chapter  II  presents  the  findings  of  a  review  of  the 
literature  base  pertaining  to  the  use  of  faces  as  a  means  of 
representing  information,  especially  financial  information.  This  review 
will  provide  the  reader  with  a  general  background  in  the  history  and  use 
of  facial  representation.  Chapter  III  presents  a  description  of  the 
methodology  used  to  accomplish  the  research  objectives.  The  methodology 
includes  an  experimental  design  to  test  a  set  research  hypotheses. 
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Chapter  IV  follows  with  an  analysis  of  experimental  results.  Chapter  V 
summarizes  the  author's  findings  and  presents  recommendations  for 
further  study. 
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II ■  Literature  Review 


Overview 

facial  representation  of  financial  data  is  a  method  of  analyzing  and 
presenting  financial  information.  It  is  not  a  well  known  method.  Very 
little  literature  exists  on  the  subject  of  facial  representation.  The 
author  of  this  literature  review  could  not  find  any  literature  dated 
after  1987  that  specifically  dealt  with  facial  representation.  Its  use, 
however,  may  prove  advantageous  over  some  of  the  other  analytical 
methods  that  are  commonly  employed  to  analyze  financial  data. 

In  this  chapter,  the  author  will  present  his  findings  on  a  review  of 
the  literature  that  pertains  to  facial  representation.  The 
justification  for  this  research  effort  is  provided,  followed  by  a 
discussion  of  the  literature.  In  the  discussion,  the  author  will 
describe  facial  representation,  present  its  potential  uses,  address  its 
good  and  bad  points,  and  identify  some  experiments  that  have  been  done 
concerning  its  use. 

Justification 

The  study  of  facial  representation  is  worthwhile  for  several  reasons. 
Financial  data  are  being  produced  every  day  by  countless  numbers  of 
sources.  Business  firms  generate  financial  statements  to  describe  the 
financial  condition  of  their  company.  The  government  generates  budgets 
and  financial  status  reports.  Government  contractors  generate  cost 
performance  data  that  they  present  to  the  government  for  review.  Most 
of  this  data  are  multivariate.  Multivariate  data  are  information 
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represented  by  several  variables,  such  as  costs,  profits,  and  budget 
variances.  Multivariate  data,  however,  are  hard  to  analyze  and  display 
because  conventional  methods  such  as  line  graphs,  bar  charts,  and  tables 
are  not  capable  of  showing  the  relationships  that  may  exist  between 
variables.  As  a  result,  much  information  is  not  discovered  because  of 
this  inefficiency.  A  graphic  that  can  display  multivariate  data  and 
show  the  relationships  between  the  variables  would  prove  advantageous 
over  conventional  methods.  Facial  representation  has  this  ability  to 
handle  multivariate  data  (7:53;  8:18). 

Facial  representation  has  been  used  to  analyze  the  financial 
condition  of  business  firms.  Because  of  its  ability  to  handle 
multivariate  data,  facial  representation  may  prove  worthwhile  in  the 
analysis  of  contractor  cost  performance  information.  This  is  the  focus 
of  this  research  study. 

Description 

Facial  representation  was  developed  by  Herman  Chemoff  in  1971 
(11:205;  13:192).  Since  then,  several  others  have  done  studies  on  the 
concept  as  well  as  conduct  experiments  concerning  its  use.  Moriarity 
records  Chemoff  s  description  of  facial  representation  as  follows: 
"Briefly  it  consists  of  representing  a  point  in  k-dimensional  space  by  a 
picture  of  a  face  whose  characteristics  are  determined  by  the  position 
of  the  point"  (11:205).  Huff,  Mahajan,  and  Black  describe  it  as  an 
approach  that: 

consists  of  representing  a  point  in  k-dimensional  space  as  a 
cartoon  of  a  face  whose  facial  features  (such  as  length  of  nose, 
curvature  of  mouth ,  size  of  eyes ,  etc . )  corresponds  to  components  of 
the  point.  (11:205) 
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Basically,  facial  representation  is  a  graphic  technique  that  matches 
specific  information  variables  with  facial  features  to  form  a  cartoon 
face.  Chemoff's  graphical  technique  allows  one  to  combine  several 
variables  into  a  picture  of  a  face.  The  face  that  is  produced  allows 
the  investigator  to  see  how  all  the  variables  fit  together  (2:361). 

Chemoff  does  not  refer  specifically  to  the  possibility  of  applying 
his  method  to  the  analysis  of  financial  information.  The  idea  that  his 
method  could  be  used  to  analyze  financial  information  is  presented  by 
others,  notably  Moriarity  (11)  and  Stock  and  Watson  (13).  Chemoff's 
methods  were  developed  to  be  used  with  any  type  of  numerical 
multivariate  data  as  a  means  of  analyzing  multivariate  information 
accurately  and  quickly  (3:548) . 

Chemoff  chose  to  use  faces  because  "People  grow  up  studying  and 
reacting  to  faces  all  the  time"  (1:363).  Benbasat  and  Dexter  suggest 
that  different  people  view  information  in  different  ways.  The  way  the 
information  is  presented  could  have  a  profound  impact  on  how  it  is  used 
(3:378-379) .  They  use  an  idea  from  Mason  and  Mitroff  to  highlight 
their  point: 

What  is  information  for  one  type  [of  decision  maker]  will  definitely 
not  be  information  for  another.  Thus  as  designers  of  MIS  [Management 
Information  Systems],  our  job  is  not  to  get  (or  force)  all  types  to 
conform  to  one,  but  to  give  each  type  the  kind  of  information  he  is 
psychologically  attuned  to  and  will  use  most  effectively.  [Mason  and 
Mitroff.  1973,  p.478]  (3:379) 

The  idea  that  the  way  information  is  presented  influences  the 
decisions  made  as  a  result  of  that  data  is  also  discussed  by  Robert 
Libby.  He  presents  three  basic  options  for  improving  decisions.  The 
first  option  is  to  change  the  information.  The  second  option  consists 
of  educating  the  person  making  the  decision  to  change  his/her  way  of 
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dealing  with  the  information.  The  final  option  is  to  replace  the 
decision  maker  with  a  model  (8:101).  Cheraoff's  technique  is  a 
combination  of  the  first  two  options  by  changing  the  way  information  is 
presented  and  then  re-educating  the  decision  maker  to  view  the 
information  in  light  of  these  faces. 

Using  Facial  Representation 

The  object  of  facial  representation  is  to  construct  a  face  based  on 
specific  information.  This  is  done  by  pairing  certain  variables  with 
specific  facial  features.  The  initial  challenge  then  is  to  determine 
which  variable  to  match  to  a  specific  facial  feature  (2:548).  Faces 
have  an  advantage  over  other  methods  of  presentation  because  they  are 
"sufficiently  rich  in  detail  to  present  a  large  number  of  variables" 
(7:206)  and  "quite  a  bit  is  known  about  the  saliency  of  features  of  a 
face,  holding  out  the  hope  that  important  financial  variables  can  be 
matched  with  salient  features  in  a  schematic  face"  (11:206).  The  first 
step,  therefore,  in  using  facial  representation  is  to  determine  which 
variables  to  match  to  specific  facial  characteristics.  Huff,  Mahajan, 
and  Black  state  that  "this  task  is  basically  one  involving  personal 
judgment  and  preference"  (7:55). 

The  second  step  for  applying  facial  representation  involves  setting 
upper  and  lower  boundaries  for  each  financial  variable  and  then  setting 
a  range  for  the  facial  feature  (7:55). 

The  literature  contains  some  general  information  on  how  to  construct 
a  face.  The  methods  used  to  construct  faces  differ  somewhat  from  user 
to  user  (7:54-55;  11:211-212).  Figure  1  shows  pictures  of  faces 
presented  in  the  literature.  Computer  programs  have  also  been  developed 
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for  the  purpose  of  constructing  faces.  Huff,  Mahajan,  and  Black 
describe  a  computer  program  that  they  employed  in  their  studies,  a 
modified  version  of  a  program  reported  in  a  paper  by  Bruckner  (7:54) . 
Moriarity  also  refers  to  the  Bruckner  program  (11:212),  as  does  Stock 
and  Watson  (13:197).  Huff  and  Black  also  describe  in  detail  a  program 
code  they  developed  along  with  the  algorithms  employed  in  developing 
that  code  (6) .  These  programs  were  developed  for  use  on  a  mainframe 
computer . 

Why  (Jse  Facial  Representation? 

Before  describing  experiments  that  have  been  accompl ished  to 
determine  the  effectiveness  of  applying  facial  representation  to 
financial  information,  it  is  appropriate  to  investigate  further  the 
advantages  and  disadvantages  of  using  it.  As  mentioned  previously,  one 
primary  rationale  for  using  facial  representation  is  its  ability  to 
present  multivariate  information.  Because  it  can  be  used  to  present 
multivariate  information,  it  allows  the  investigator  to  study  several 
related  variables  at  one  time.  As  a  graphical  technique,  it  can  help 
the  investigator  study  the  relationships  between  variables  and  identify 
any  trends  that  may  be  developing  between  the  variables  (8:118;  11:205). 
A  multivariate  graphic  helps  the  investigator  see  relationships  between 
variables  that  are  not  detectable  using  a  two-dimensional  graphic.  This 
is  true  because  variables  which  have  been  previously  shown  on  two 


9 


A.  Moriarity  and  Roach  (1977)  - 
from  Libby  (8:119) 


Figure  1.  Examples  of  Faces  in  Literature. 
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separate  graphics  can  be  shown  as  one  variable  on  a  multivariate 
graphic.  Then,  as  the  value  of  one  variable  changes,  the  effect  on  the 
other  variable  is  readily  apparent  (7:56). 

Facial  representation  has  several  advantages  over  other  multivariate 
graphic  techniques.  Its  unique  way  of  presentation  is  one  specific 
advantage.  Chemoff  highlights  the  advantage  as  follows:  "People  grow 
up  studying  and  reacting  to  faces  all  of  the  time.  Small  and  barely 
measurable  differences  are  easily  detected  and  evoke  emotional  reactions 
from  a  long  catalogue  buried  in  the  memory”  (1:363).  Facial 
expressions  prompt  an  emotional  response  from  humans.  In  addition, 
humans  have  the  ability  to  detect  even  the  slightest  change  in  a  facial 
expression.  This  acts  as  a  big  advantage  when  faces  are  used  to  portray 
financial  information.  When  one  looks  at  a  face,  he/she  can  easily  pick 
out  the  major  characteristics  that  are  portrayed  by  that  face.  This 
ability  enables  that  person  to  gain  quickly  an  understanding  of  the 
information  represented.  After  a  more  detailed  study  of  the  faces,  that 
person  would  be  able  to  pick  out  fine  details  and  correlations  that  are 
present,  and  which  were  not  apparent  at  first  glance.  This  would  be 
done  in  such  manner  that  "The  awareness  of  these  [fine  details  and 
correlations]  does  not  drive  out  of  the  mind  the  original  major 
impressions"  (2:364).  Facial  representation,  therefore,  enhances  the 
investigator's  ability  to  perceive  the  information  and  process  it. 
Chemoff  adds:  "The  major  advantage  to  be  derived  from  using  the  faces 
should  be  in  the  heightened  qualitative  awareness  of  which  numerical 
calculations  are  relevant"  (1:364). 

Because  people  are  familiar  with  faces,  facial  representation  may 
help  them  understand  information  that  tney  otherwise  might  not  be  able 
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to  grasp.  The  manager  who  has  trouble  comprehending  financial 
information  as  it  is  normally  presented  (tables,  line  graphs,  etc.)  may 
be  able  to  relate  to  a  series  of  faces  representing  that  information, 
and  as  a  result,  gain  an  understanding  of  what  that  information  is 
trying  to  express.  With  a  better  understanding  of  the  information  the 
manager  will  be  able  to  make  better  decisions  (11:206). 

Another  advantage  facial  representation  offers  is  its  ability  to 
graph  several  variables  at  one  time.  Certain  studies  have  shown  that  as 
many  as  13  variables  could  be  displayed  on  a  single  face  (11:206). 

Table  1  lists  variables  employed  in  constructing  a  face  along  with  the 
facial  feature  each  variable  has  been  matched  with. 

One  other  advantage  in  using  facial  representation  to  portray 
financial  information  that  is  worth  mentioning  is  that  the  method  is 
inexpensive  and  easy  to  use.  The  technology  for  drawing  faces  has 
already  been  developed.  Computer  programs  have  been  developed  for 
mainframe  computers  that  will  draw  the  faces,  eliminating  the  hassle  of 
drawing  faces  by  hand  (8:118) . 

One  of  facial  representation's  advantages  is  also  one  of  its 
disadvantages.  This  is  its  uniqueness.  Because  the  cartoon  faces  are 
also  funny  looking,  it  may  be  hard  for  people  to  take  them  seriously. 
This  may  diminish  their  effectiveness  (8:120).  Another  potential 
disadvantage  is  that  "the  relative  importance  of  certain  variables  may 
be  exaggerated  because  of  the  facial  features  to  which  they  have  been 
assigned"  (7:57-58).  People  may  perceive  the  importance  of  certain 
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TABLE  1 


List  of  Sample  Financial  Variable/Facial  Feature  Matches  (7:56) 


FINANCIAL  VARIABLE 

Return  on  Assets 
Debt  Service 

Cash  flow 
Capitalization 
Current  Ratio 
Cash  turnover 
Receivables  turnover 
Inventory  turnover 
Sales  per  dollar 
working  capital 
Retained  earnings/ 
total  assets 
Total  assets 


FACIAL  FEATURE 

Curvature  of  mouth 
Eccentricity  of  lower 
ellipse  (face) 
Eccentricity  of  eyes 
Half-length  of  eyes 
Ear  diameter 
Angle  of  eyebrow 
Position  of  pupils 
Length  of  eyebrow 
Eccentricity  of  upper 
ellipse  (face) 

Length  of  nose 

Width  of  nose 


facial  features  differently.  One  person  may  assign  more  importance  to 
the  nose  than  the  mouth,  while  another  may  assign  more  importance  to  the 
mouth  than  the  nose. 

Feasibility  Studies 

Much  of  the  literature  about  facial  representation  of  financial 
variables  pertains  to  experiments  that  were  conducted  to  determine  the 
feasibility  of  using  faces  to  portray  financial  information.  In  one 
such  experiment,  participants  were  to  detect  a  change  in  the  financial 
condition  of  a  company  (13:194).  Another  experiment  had  students 
analyzing  companies  to  determine  if  they  qualified  as  candidates  for 
bankruptcy  (11:206).  The  intent  of  both  experiments  was  to  determine 
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whether  students  could  analyze  the  pertinent  information  more  accurately 
and  quickly  with  facial  representation  than  without  it. 

The  methodology  used  to  conduct  the  above  experiments  was  similar  for 
both  experiments.  In  each  case,  the  researchers  sought  out  people 
having  varied  levels  of  accounting  experience.  The  experiments  were 
conducted  with  groups  of  twenty  to  thirty  people.  The  participants  were 
given  information  on  twenty  to  twenty-two  different  firms.  The 
information  for  each  firm  was  contained  on  one  8  1/2"  by  11"  sheet  of 
paper.  The  information  was  presented  with  and  without  faces.  Moriarity 
differed  from  Stock  and  Watson  by  presenting  the  faces  with  and  without 
an  explanation  of  what  each  facial  feature  represented.  Stock  and 
Watson  did  not  include  such  an  explanation.  In  both  cases,  a  number  was 
assigned  to  each  of  the  firms.  The  collection  of  the  financial 
information  representing  the  firms  was  placed  in  9"  by  12"  packets  and 
then  distributed.  Instructions  were  given  orally.  The  information  in 
the  packet  also  included  a  statement  that  "The  firms  included  in  your 
packet  were  determined  randomly  so  that  your  answer  may  differ  from  that 
of  your  neighbor"  (13:196).  This  statement  was  not  true.  All 
participants  received  the  same  firms.  Participants  received  five 
minutes  of  instruction  and  were  given  20  minutes  to  complete  the  study 
(11:207-213;  13:196). 

The  results  of  Moriarity 's  experiment  are  shown  in  Table  2.  In  his 
experiment  participants  analyzed  the  information  more  accurately  and 
more  quickly  with  the  use  of  faces  than  without.  In  both  cases  this 
difference  was  significant  at  the  0.05  level.  Moriarity  used  the  data 
for  both  decision  accuracy  and  decision  speed  to  calculate  a  score  for 
each  condition.  The  score  was  equal  to  the  percentage  of  correct 
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answers  divided  by  the  time  to  complete  the  task.  This  difference 
between  scores  also  favored  the  faces  mode  and  was  significant  at  the 
0.05  level.  When  the  information  was  presented  in  the  form  of  faces 
along  with  an  explanation  of  what  the  faces  represented,  participants 
analyzed  the  information  more  accurately  with  the  use  of  the  faces  with 
the  explanation  than  with  the  use  of  faces  without  explanation,  although 
this  difference  was  not  significant  at  the  0.05  level.  The  use  of  faces 
without  explanation  was  quicker  than  the  use  of  faces  with  explanation. 
This  difference  was  significant  at  the  0.05  level.  The  difference  in 
score  also  favored  the  use  of  faces  without  explanation.  This 
difference  was  significant  at  the  0.05  level  (11:213-218). 

In  Stock  and  Watson's  experiment,  participants  were  also  more 
accurate  in  their  analysis  when  faces  were  used.  The  results  of  their 
experiment  are  also  shown  in  Table  2.  The  difference  in  accuracy  was 
significant  at  the  0.05  level.  This  difference  did  not  vary 
significantly  based  on  the  level  of  the  participant's  accounting 
knowledge  (13:198).  Stock  and  Watson  also  tested  decision  times. 

Their  findings  in  this  area  did  not  support  Moriarity's  findings  in  that 
the  difference  in  decision  times  between  the  two  modes  was  not 
significant  (13:199). 

Moriarity  drew  this  conclusion  from  his  experiment: 

While  the  evidence  from  the  experiment  suggests  that  schematic  faces 
are  an  efficient  means  of  communi eating  financial  information,  there 
are  several  areas  related  to  their  construction  and  use  which  need 
further  research.  (13:222) 

These  additional  areas  of  research  include  investigating  different 
financial  characteristics,  changing  the  financial  variable/facial 
feature  pairings,  and  examining  the  effect  of  training  with  the  faces  to 
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determine  whether  it  might  be  less  expensive  to  train  people  in  the  use 
of  faces  as  opposed  to  other  techniques  (13:222-223). 

Stock  and  Watson  report  their  conclusions  based  on  human  judgment 
accuracy.  "The  results  indicate  that  human  judgement  accuracy  can  be 
influenced  by  the  report  format  used  for  displaying  accounting 
information"  (13:200).  They  also  consider  that  "The  implication  of 
these  results  is  that  accountants  might  consider  changing  or  augmenting 
the  format  for  displaying  accounting  information  as  one  option  for 
improving  human  judgment  accuracy"  (13:200). 

A  third  experiment  was  performed  after  Moriarity  and  Stock  and 
Watson.  Mackay  and  Villarreal  (MV)  set  out  to  test  the  validity  of 
Moriarity's  and  Stock  and  Watson's  findings.  Their  research  focused  on 
three  main  issues: 

1.  "The  validity  of  the  conclusion  that  faces  do  better  than 
tabular  displays"  (9:536), 

2.  "The  identification  of  situations  where  one  display 
procedure  may  be  expected  to  be  superior  to  the  other"  (9:537), 
and 

3.  "The  role  of  individual  differences  in  determining  the 
relative  success  of  using  facial  or  tabular  displays"  (9:537). 

The  basis  for  their  argument  in  issue  one  was  that  participants  in 
Moriarity's  experiment  had  very  little  understanding  of  what  the 
financial  ratios  represented.  MV  believed  that  in  order  to  obtain  a 
more  meaningful  comparison  participants  should  have  had  some  experience 
working  with  the  financial  ratios  and  that  the  ratios  should  have  been 
defined  carefully. 
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TABLE  2 


Summary  of  Experiment  Results 


A.  Moriarity  (1979) 


Mode  Of  Presentation 

Faces  without 

Faces  With 

Participants 

Explanation 

Explanation 

Ratios 

Students 

Average  Errors 

7.313 

7.092 

8.620 

Average  Time  (sec) 

218.88 

371.88 

480.96 

Score 

11.617 

6.814 

5.211 

Experts 

Average  Errors 

5.75 

6.20 

7.40 

Average  Time  (sec) 

280.08 

376.92 

344.88 

Score 

9.652 

7.185 

7.056 

Score  -  percent  correct/time  (in  hours) 


B.  Stock  and  Watson  (1984) 


Percent  Correct  Answers 

Level  Of  Experience 

Using  Faces* 

Using  Ratios 

Elementary 

51.32 

47.35 

Intermediate  I 

50.00 

45.60 

Intermediate  II 

50.68 

45.08 

Experts 

52.38 

42.85 

Overall 

50.70 

45.67 

*  Faces  provided  without  explanation. 
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MV's  concern  in  issue  two  is  that  both  experiments  (Moriarity  and 
Stock  and  Watson)  focused  on  determining  which  was  the  better  method, 
faces  or  ratios.  The  focus,  they  believed,  should  be  on  determining  the 
situations  where  one  method  outperforms  the  other.  This  could  be  done 
by  "looking  at  the  qualities  attributed  to  graphic  and  tabular  methods 
and  then  considering  circumstances  where  these  qualities  might  make  a 
difference"  (9:537). 

MV’s  third  issue  suggests  that  the  field  of  psychological  literature 
that  deals  with  the  responses  of  subjects  to  facial  displays  should  be 
examined  and  applied  to  the  area  of  representing  financial  data  with 
faces.  MV  states  that  "one  of  the  primary  conclusions  of  this 
literature  is  that  individuals  do  differ  in  their  responsiveness  to 
facial  cues"  (9:537). 

To  address  his  issues  MV  also  conducted  an  experiment  in  which  he  had 
participants  evaluate  whether  a  particular  firm  was  a  candidate  for 
bankruptcy.  Participants  were  shown  financial  data  in  both  display 
modes,  faces  and  ratios.  The  financial  data  were  obtained  from 
different  firms,  half  of  which  had  file  for  bankruptcy.  MV  used  a 
sample  of  98  MBA  students,  most  of  which  had  prior  work  experience  in 
the  use  of  ratios  (9:538-539). 

MV's  results  did  not  support  those  obtained  by  Moriarity  and  Stock 
and  Watson.  The  difference  in  the  amount  of  correct  answers  favored  the 
faces  mode  (difference  in  mean  scores  was  0.23),  however  this  difference 
was  not  significant.  MV's  results  did  support  Moriarity's  results  in 
that  the  use  of  faces  resulted  in  faster  decision  times.  These  results 
supported  MV's  premise  in  his  first  issue,  that  the  lack  of  experience 
in  using  ratios  had  an  impact  on  the  favoritism  that  the  faces  mode 
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experienced  in  the  previous  experiments.  MV  compensated  for  this  lack 
of  experience,  and  as  a  result  the  faces  mode  was  not  as  favored  (9:540- 
541) . 

The  results  of  MV’s  experience  also  supported  their  arguments  in 
their  other  two  issues.  In  addressing  the  second  issue,  MV  found  that 
the  ratios  did  a  better  job  predicting  bankruptcy  while  the  faces  did  a 
better  job  predicting  successful  firms.  In  both  cases,  the  differences 
were  significant.  In  addressing  the  third  issue,  MV  investigated 
individual  differences  in  responses  to  both  modes  of  presentation.  The 
only  significant  variable  that  appeared  was  gender.  Women  were  found  to 
process  facial  cues  more  successfully  than  men  (9:541-544). 

Summary 

Face  analysis  may  be  an  effective  tool  for  analyzing  financial 
information.  Face  analysis  offers  several  advantages  over  other 
techniques.  Its  ability  to  handle  multivariate  data  allows  the 
investigator  to  see  the  relationship  and  trends  occurring  among  several 
variables.  The  familiarity  that  people  have  with  faces  makes  it  an 
effective  analytical  tool  for  those  individuals  not  familiar  with 
financial  reporting  concepts. 

Face  analysis  does  have  disadvantages.  Because  of  its  use  of  cartoon 
faces,  s<xne  people  may  not  take  it  seriously.  The  possibility  that  seme 
people  may  misinterpret  the  significance  of  some  of  the  financial 
variable/facial  feature  pairings  may  also  decrease  its  effectiveness. 

Experiments  have  been  conducted  that  show  that  the  use  of  faces  in 
analyzing  financial  data  is  more  effective  than  some  of  the  common 
analytical  means.  The  results  of  these  experiments  were  similar  for 
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people  with  little  knowledge  pertaining  to  the  analysis  of  financial 
data  and  for  people  with  much  knowledge  in  this  area. 

The  conclusion  that  the  literature  presents  is  that  facial 
representation  may  be  a  good  tool  for  analyzing  and  presenting  financial 
data.  Additional  research  involving  facial  representation  of  financial 
information  is  encouraged. 
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III.  Methodology 


The  primary  method  that  was  employed  in  the  execution  of  this 
research  study  is  experimentation. 

Justification  of  Experimental  Approach 

Up  to  this  point,  facial  representation  has  never  been  used  to 
portray  and  analyze  Air  Force  contractor  cost  performance  data. 
Sufficient  data  do  not  exist  that  can  be  used  to  determine  whether 
facial  representation  is  an  effective  decision  making  tool  when  applied 
to  such  data.  Therefore,  this  data  must  be  produced.  Experimentation 
is  an  effective  means  of  producing  this  data.  It  can  be  done  within  the 
time  limitations  placed  on  this  research  study.  It  also  allows  the 
investigator  to  control  the  independent  variables,  user  background  and 
mode  of  presentation,  described  in  Chapter  I. 

Task  to  be  Accomplished 

As  described  through  the  use  of  the  investigative  questions  in 
Chapter  I,  the  task  to  be  accomplished  was  composed  of  four  major  areas: 

1.  Obtaining  or  developing  a  PC-based  computer  program  for  drawing 
the  faces, 

2.  Selecting  CPR  variables  to  be  analyzed  and  matching  those 
variables  to  facial  features, 

3.  Determining  the  usefulness  and  effectiveness  of  facial 
representation  as  a  decision  making  tool,  and 

4.  Determining  whether  a  person’s  level  of  experience  has  an  impact 
on  the  usefulness  of  the  faces  method. 
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Upon  initial  evaluation  of  experimental  results,  a  fifth  task  area  was 
added: 

5.  Determining  the  impact  of  both  learning  and  training  on  the 

effectiveness  of  the  faces  method. 

Task  areas  three  through  five  were  accomplished  through  experimentation 
and  survey.  Task  areas  one  and  two  had  to  be  accomplished  first  in 
order  for  the  experiment  to  be  developed.  The  remainder  of  this  chapter 
describes  the  methods  used  to  accomplish  the  above  tasks. 

Computer  Program  Development 

Before  the  experiment  was  developed,  a  computer  program  able  to  draw 
the  faces  and  that  is  also  PC-based  had  to  be  obtained  or  written.  The 
program  had  to  be  PC-based  rather  than  mainframe-based  for  two  reasons. 
First,  one  intent  of  this  research  study  is  the  development  of  the  faces 
concept  for  use  as  a  tool  in  Air  Force  cost  performance  measurement. 

For  it  to  be  used  extensively  as  a  tool,  it  needs  to  be  inexpensive  and 
obtainable.  A  PC-based  program  satisfies  those  requirements.  Second,  a 
PC-based  program  was  required  to  conduct  the  research  experiments.  A 
PC-based  program  significantly  aids  in  data  entry,  data  selection,  and 
data  analysis.  The  investigator  conducted  a  search  to  determine  whether 
a  PC  based  computer  program  already  exists.  To  date  no  such  PC-based 
program  has  been  discovered,  although  not  all  avenues  have  been 
searched.  The  investigator  had  obtained  a  written  code  for  a  program 
that  is  not  PC-based  (6).  A  few  other  programs  were  also  available  for 
mainframe-based  computers.  Because  a  PC-based  program  does  not  exist, 
the  investigator  developed  one  using  Microsoft  QuickBasic  (10) . 
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Appendix  A  contains  a  description  of  this  program,  called  CPR  FACES. 

The  computer  code  for  CPR  FACES  is  also  provided  in  Appendix  A. 

Variable  Selection/Facial  Feature  Pairing 

Once  the  program  was  developed,  the  investigator  paired  contractor 
cost  performance  data  variables  to  appropriate  facial  characteristics. 
Since  the  choice  of  pairings  is  basically  a  function  of  "personal 
judgment  and  preference"  (7:55)  the  actual  pairings  may  not  represent 
the  optimal  pairing  possible.  A  study  of  the  effects  of  different  cost 
performance  variable/facial  characteristic  pairings  will  be  left  for 
possible  future  research.  This  information  was  put  into  the  computer 
program. 

Research  Hypotheses 

As  mentioned  previously,  task  areas  three  through  five  were 
accomplished  through  experimentation  and  survey.  This  section  describes 
the  hypotheses  that  were  used  in  this  study  to  accomplish  those  task 
areas.  The  following  section  describes  the  experiment  and  survey  used 
to  test  the  hypotheses. 

Effectiveness/Usefulness  Hypotheses.  To  determine  the  degree  of 
decision  making  effectiveness  that  can  be  achieved  when  faces  are  used 
to  analyze  (PR  data,  several  hypothesis  were  developed.  Decision  making 
effectiveness  is  a  function  of  both  decision  speed  and  decision 
accuracy.  A  separate  hypothesis  was  developed  to  test  both  of  these 
factors.  A  third  hypothesis  was  designed  to  test  specifically  for 
decision  making  effectiveness.  This  hypothesis  utilizes  the  data 
obtained  from  testing  the  first  two  hypotheses,  that  of  decision  speed 
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and  decision  accuracy.  A  fourth  hypothesis  deals  with  a  specific  area 
of  decision  making  effectiveness,  the  analysis  of  trends.  To  determine 
a  degree  of  usefulness,  one  primary  factor  was  considered,  decision 
making  confidence.  The  assumption  behind  this  hypothesis  was  that 
individuals  would  not  use  a  specific  decision  making  tool  if  they  were 
not  confident  making  decisions  when  using  that  tool .  The  hypotheses  are 
listed  below. 


1.  Individuals  will  take  less  time  evaluating  data  in  the 
faces  format  than  they  will  evaluating  data  in  the  tabular 
format. 

2.  Individuals  will  be  more  accurate  evaluating  data  in  the 
faces  format  than  they  will  evaluating  data  in  the  tabular 
format . 

3.  Individuals  will  be  more  effective  making  decisions  when 
evaluating  data  in  the  faces  format  than  they  will  evaluating 
data  in  the  tabular  format. 

4.  When  attempting  to  evaluate  trends  in  data,  individuals 
will  be  more  effective  when  using  the  faces  format  them  they 
will  using  the  tabular  format. 

5.  Individuals  will  feel  as  confident  using  the  faces 
presentation  mode  as  they  will  using  the  tabular  presentation 
mode. 

Impact  of  Experience  Level  on  Decision  Making  Effectiveness 
Hypothesis.  One  hypothesis  was  created  to  determine  the  impact  that  an 
individual's  experience  level  in  the  analysis  of  CPR  data  had  on 
decision  making  effectiveness.  The  hypothesis  is: 


6.  Individuals  without  experience  in  analyzing  cost 
performance  data  will  score  the  same  on  decision  making 
effectiveness  when  analyzing  face  data  as  those  individuals 
with  cost  performance  report  experience. 
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Impact  of  Learning  and  Training  Hypotheses.  The  next  hypothesis  was 


developed  after  initial  evaluation  of  experimental  results.  The 
investigator  noted  that  decision  making  effectiveness  scores  on  the 
second  half  of  the  experiment  appeared  to  be  greater  than  on  the  first 
half  of  the  experiment.  The  hypothesis  was  used  to  test  if  learning  was 
occurring  during  the  experiment. 

7.  During  the  experiment  learning  occurred  in  such  a  way  that 
decision  making  effectiveness  was  greater  in  the  bottom  half  of 
the  experiment. 

The  above  hypothesis  led  to  the  development  of  further  hypotheses 
that  were  used  to  determine  whether  training  participants  in  the  use  of 
faces  would  result  in  greater  decision  making  effectiveness  and  greater 
usefulness,  (again,  using  decision  making  confidence  as  a  factor  for 
measuring  usefulness) .  To  test  for  the  impact  of  training  on  decision 
making  effectiveness  and  usefulness  five  hypotheses  were  tested  in  this 
study.  The  instruction  mentioned  in  these  hypotheses  refers  to  five 
minutes  of  oral  teaching  that  was  given  to  experiment  participants  to 
instruct  them  on  how  the  faces  should  be  interpreted.  The  hypotheses 
that  were  tested  are: 

8.  Individuals  who  received  a  five  minute  instruction  on  the 
use  of  faces  will  answer  the  faces  related  questions  faster 
than  those  who  did  not  receive  the  instruction. 

9.  Individuals  who  received  a  five  minute  instruction  on  the 
use  of  faces  will  answer  the  faces  related  questions  more 
accurately  than  those  who  did  not  receive  the  instruction. 

10.  Individuals  who  received  a  five  minute  instruction  on  the 
use  of  faces  will  score  higher  in  decision  making  effectiveness 
when  answering  the  faces  related  questions  than  those  who  did 
not  receive  the  instruction. 
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11.  Individuals  who  received  a  five  minute  instruction  on  the 
use  of  faces  will  be  more  confident  making  decisions  using 
faces  than  those  who  did  not  receive  the  instruction. 

12.  Individuals  who  received  the  five  minute  instruction  on 
the  use  of  faces  will  more  likely  to  use  the  faces  format  in 
their  jobs,  if  given  the  opportunity,  than  those  who  did  not 
receive  the  instruction. 

Experiment  Construction 

Ideas  for  the  experiment  framework  were  obtained  from  experiments  by 
Moriarity  (2:207-213),  Stock  and  Watson  (13:194-199)  and  Grigware  (5:16- 
25).  The  focus  of  interest  is  contractor  cost  performance  data. 
Contractor  performance  data  were  fabricated  for  eight  fictitious 
contracts.  Participants  were  provided  cost  performance  report  data  for 
each  of  these  contracts  in  one  of  two  ways: 

1.  Tabular  Form  listing  Budgeted  Cost  of  Work  Scheduled  (BCWS) , 
Budgeted  Cost  of  Work  Performed  (BCWP) ,  Actual  Cost  of  Work  Performed 
(ACWP) ,  Schedule  Variance,  and  Cost  Variance;  or 

2.  Facial  Representation  -  A  set  of  faces  representing  contractor 
performance . 

Each  participant  received  four  sets  of  data  using  the  tabular  form 
and  four  sets  of  data  using  faces.  The  data  sets  were  randomly 
distributed  such  that  while  each  participant  received  the  same  eight 
contracts,  the  distribution  of  the  presentation  modes  of  the  sets  were 
different. 

A  QuickBASIC  program  was  developed  for  the  experiment.  A  description 
of  the  program  is  provided  in  Appendix  B.  The  experiment  was  done  on 
computer  instead  of  using  paper  handouts  to  add  validity  to  the 
experiment .  Since  the  CPR  faces  concept  is  intended  to  be  a  computer 
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based  tool,  a  computer  based  experiment  will  provide  more  realistic 
scenarios.  The  experiment  program  consisted  of  four  main  areas.  The 
first  program  area  consisted  of  an  introduction  screen,  which  included  a 
narrative  about  the  experiment.  The  second  area  was  designed  to  ask  the 
participant  a  few  background  questions  such  as  Air  Force  Specialty  Code 
(AFSC)  and  experience.  This  information  was  used  to  characterize 
participants  by  background,  allowing  the  investigator  to  determine  the 
impact  of  user  background  on  experimental  results.  In  the  third  area, 
the  participant  was  shown  a  screen  with  three  faces  and  three 
corresponding  tabular  data  sets.  Che  data  set  and  face  depicted  neutral 
contract  performance  (contract  on  cost  and  on  schedule) .  Another  data 
set  and  face  depicted  very  good  contract  performance  (well  ahead  of 
schedule  and  well  under  cost) .  The  third  data  set  depicted  very  bad 
contract  performance  (well  behind  schedule  and  well  under  cost).  The 
last  eight  screens  comprised  the  fourth  area.  This  area  was  the  main 
experiment  section.  In  this  area,  participants  were  shown  cost 
performance  measurement  data  in  the  form  of  faces  and  in  tabular  form. 
They  were  also  asked  questions  related  to  the  data  on  the  screen. 

Experiment  Design.  Each  of  the  eight  screens  in  the  fourth  area  were 
designed  to  represent  the  performance  of  one  contract.  Four  CPR  data 
sets  were  generated  for  each  screen;  these  four  data  sets  will  be  called 
a  contract  data  group.  Thirty-two  data  sets  were  generated  for  the 
experiment,  or  a  total  of  eight  contract  data  groups.  Four  of  the 
contract  data  groups  included  information  for  three  sub-elements  of  the 
contract  and  information  for  overall  contract  performance.  These 
contract  data  groups  were  designed  to  test  the  participant’s  ability  to 
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analyze  actual  contract  perf ormance .  The  other  four  contract  data 
groups  included  information  for  the  overall  performance  of  the  contract 
over  four  periods  of  time.  These  contract  data  groups  were  designed  to 
test  the  participant's  ability  in  analyzing  trends  in  contract 
performance . 

The  eight  contract  data  groups  were  then  assembled  by  generating 
eight  different  combinations  of  the  groups.  Each  overall  experiment 
data  set  (consisting  of  all  eight  data  groups)  was  copied  to  a  floppy 
disk  containing  the  experiment  program.  Thirty-two  floppy  disks  were 
used.  Each  overall  data  set  was  included  on  four  floppy  disks.  The 
thirty-two  disks  were  randomly  sorted  and  distributed.  As  a  result,  all 
participants  were  shown  four  groups  of  CPR  data  in  the  form  of  faces  and 
four  sets  in  the  form  of  tables.  All  eight  contract  data  groups  were 
shown  to  each  participant;  however,  the  program  was  designed  such  that 
while  one  person  saw  a  particular  group  in  the  form  of  faces,  another 
person  would  see  the  same  group  in  the  form  of  tables.  This  way,  each 
participant  had  the  opportunity  to  use  both  faces  and  tables. 

The  contract  data  groups  were  also  distributed  to  act  us  a  control  on 
itself.  Since  each  group  was  shown  using  both  faces  and  tables,  no 
group  had  an  advantage  over  another.  The  contract  data  groups  were  also 
distributed  such  that  a  particular  group  might  appear  at  any  stage  in 
the  experiment.  One  person  would  see  contract  data  group  1,  for 
instance,  in  the  first  of  the  eight  screens.  Another  might  see 
contractdata  group  1  in  screen  five.  Appendix  C  provides  a  listing  of 
the  data  used  for  the  experiment.  Appendix  D  shows  the  experiment  test 
scenario. 
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Experiment  Questions.  For  each  contract  data  group  designed  to  test 
participant's  ability  to  analyze  contract  performance,  participants  were 
asked  four  questions  to  test  their  ability  to  draw  information  and  make 
decisions  from  the  data.  The  questions  are: 

1)  Is  the  contractor  behind  or  ahead  of  schedule? 

2)  Is  the  contractor  over  or  under  cost? 

3)  What  elements  (from  the  three  sub-elements)  are  over  cost? 

4)  What  elements  are  behind  schedule? 

For  each  contract  data  set  designed  to  test  the  participant's  ability 
to  analyze  trends  in  contract  performance,  the  following  two  questions 
were  asked: 

1)  What  is  the  trend  in  cost  performance  (getting  better, 
getting  worse,  or  no  trend)? 

2)  What  is  the  trend  in  schedule  performance  (getting  better, 
getting  worse,  or  no  trend)? 

The  experiment  program  was  designed  to  record  the  time  that  the 
participant  took  on  each  screen.  This  was  devised  to  record  the  amount 
of  time  it  took  each  participant  to  analyze  the  data  and  answer 
questions  based  on  that  data.  This  information  was  recorded  in  order  to 
determine  a  measure  of  decision  making  effectiveness,  calculated  using 
both  the  number  of  correct  answers  and  the  time  needed  to  answer 
questions. 

Experiment  Administration 

The  experiment  was  administrated  to  groups  of  2  to  12  people.  The 
group  received  five  minutes  of  oral  explanation  and  instructions  dealing 
with  the  actual  operation  of  the  experiment  program.  Each  participant 
was  given  an  information  sheet  listing  some  key  definitions  as  well  as 
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the  facial  feature,  cost  performance  pairings  used  in  the  faces  program 
(Figure  2) ,  and  another  sheet  that  was  exactly  the  same  as  the  computer 
screen  mentioned  previously  which  list  faces  and  data  sets  depicting 
good,  bad,  and  neutral  performance  (Figure  3). 

Population.  Participants  were  selected  from  current  AFIT  graduate 
programs.  The  investigator  had  hoped  to  include  students  from  AFIT 
Professional  Continuing  Education  (PCE)  courses;  however,  due  to  the 
amount  of  time  it  took  construct  the  program  and  the  difficulty  of 
scheduling  computer  rooms  with  PCE  times,  this  was  not  possible.  The 
experiment  requires  that  each  participant  have  at  least  a  general 
knowledge  of  cost  performance  measurement  to  the  extent  that  they  were 
at  least  familiar  with  the  term.  To  accomplish  this,  participants  were 
limited  to  the  following  groups: 

1 .  AFIT  students  in  either  the  Graduate  Systems  Management  (GSM) 
program,  the  Graduate  Contracting  Management  (GCM)  program,  and  the 
Graduate  Cost  Analysis  (GCA)  program. 

2.  AFIT  PCE  students  attending  cost  performance  measurement  related 
courses.  Again,  due  to  the  above  constraints,  this  was  not  possible. 

Experiments  were  conducted  in  groups  of  2  to  12  participants.  This 
size  was  limited  due  to  the  number  of  available  computers .  Because  it 
was  more  feasible  to  choose  classes  or  groups  that  were  already  formed, 
the  sample  was  not  rarriom.  Each  member  of  the  population  did  not  have 
an  equal,  non-zero  chance  of  being  selected. 
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Information  Sheet 


The  following  information  may  prove  helpful  when  answering  the 
questions.  Please  take  a  few  moments  to  look  over  this  sheet 
before  starting  and  refer  back  to  this  sheet  if  necessary  during 
the  experiment: 

DEFINITIONS: 

BCWS  -  Budgeted  Cost  of  Work  Scheduled  -  the  amount  of  work  that 
should  have  been  completed  to  date  and  the  amount  it  should  have 
cost. 

BCWP  -  Budgeted  Cost  of  Work  Performed  -  the  amount  of  work  that 
has  actually  been  completed  and  the  amount  that  work  should 
have  cost. 

ACWP  -  Actual  Cost  of  Work  Performed  -  the  actual  cost  of  the 
work  that  has  been  completed. 

COST  VARIANCE  =  BCWP  -  ACWP  If  positive,  contractor  is  under 
cost 

SCHEDULE  VARIANCE  =  BCWP  -  BCWS  If  positive,  contractor  is  ahead 
of  schedule 


The  following  facial  characteristic  -  cost  performance  variables 
are  represented: 

Cost  Variance  -  eyebrow  position 
Schedule  Variance  -  pupil  position 

Cost  Performance  Index  -  measure  of  cost  performance  equal  to 
BCWP/ACWP  -  curvature  of  the  mouth 

Schedule  Performance  Index  -  measure  of  schedule  performance 
equal  to  BCWP/BCWS  -  nose  length 

TCPI  -  "To  Completion  Performance  Index"  -  measure  of  ability  to 
complete  work  on  schedule  -  shape  of  face 

The  following  screen  (after  the  information  screen)  provides  some 
sample  faces  with  accompanying  tabular  data. 


Figure  2  -  Information  Sheet 
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Cost  Performance  Information 


BCUS 

2080 

10000 

4000 

BCWP 

4000 

4000 

4000 

ACWP 

nmin 

£JQ 00 

7008 

4008 

DAT 

Dnv 

15000 

15000 

15000 

Figure  3  -  Examples  of  Faces 
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Criteria  for  Analysis 


The  results  of  the  analysis  using  these  two  methods  of  presentation 
will  be  compared  to  the  actual  performance  that  the  data  represents. 

This  comparison  was  used  to  compute  a  measure  of  effectiveness  in 
decision  making  using  facial  representation  versus  that  of  using  the 
data  presented  in  tabular  form.  Effectiveness  will  be  defined  as 
accuracy  of  decision  (number  of  questions  answered  correctly)  per  unit 
time  taken  to  make  that  decision  (in  minutes). 

Tests  for  Decision  Making  Effectiveness.  Hypotheses  one  through  four 
were  tested  using  data  obtained  from  the  above  experiment  questions. 
Decision  speed  was  based  on  the  amount  of  time  taken  to  complete  a 
particular  question  set.  Decision  accuracy  was  based  on  the  number  of 
correct  answers  achieved.  Decision  making  effectiveness  scores  were 
based  on  both  decision  speed  and  decision  accuracy.  Hypothesis  four, 
dealing  with  trend  determination,  was  based  only  on  those  questions 
pertaining  to  the  analysis  of  trends. 

Tests  for  Decision  Making  Confidence.  At  the  end  of  the  computerized 
experiment,  participants  were  given  an  eight  statement  survey  (Figure  4) 
designed  to  measure  the  following  factors. 

1.  The  confidence  that  the  participant  had  in  making 
decisions  using  data  presented  in  the  form  of  faces  and  in 
tabular  form  (statements  one  and  two) ; 

2.  The  perception  of  the  participant  of  the  accuracy  of  both 
modes  of  presentation  (statements  three  and  four) ; 

3.  The  perception  of  the  participant  of  the  ease  in 
understanding  information  when  presented  in  the  form  of  faces 
and  in  tabular  form  (statements  five  and  six); 

4.  The  perception  of  the  participant  on  whether  or  not  he  /she 
felt  that  he /she  answered  more  questions  correctly  using  the 
faces  format  (statement  seven):  and. 
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5.  Whether  or  not  the  participant  would  use  the  faces  format 
in  a  job  experience  if  he/she  had  opportunity  to  do  so 
(statement  eight) . 

The  survey  used  a  five  point  Likert  scale  to  measure  responses  to  the 
survey  statement.  A  response  of  five  indicated  that  the  person  strongly 
agreed  with  the  statement,  a  response  of  one  indicated  that  the 
participant  strongly  disagreed  with  the  statement. 

Hypothesis  five  was  tested  using  the  mean  number  of  answers  to  parts 
one,  two  and  three  above  for  the  faces  format  (survey  questions  one, 
three,  and  five)  and  the  mean  number  of  answers  to  parts  one,  two,  and 
three  for  the  tabular  format  (survey  questions  two,  four,  and  six. 

The  answers  to  part  four  (survey  question  seven)  were  added  up  to 
obtain  a  mean  value.  This  value  gives  an  indication  as  to  whether 
participants  felt  they  made  good  decisions  using  faces  as  compared  to 
the  decisions  they  made  using  tables.  A  mean  value  of  3.0  would 
indicate  that  they  felt  that  decisions  made  with  faces  were  as  good  as 
those  made  with  tables. 

The  answers  to  part  five  (survey  question  eight)  were  added  up  to 
obtain  a  mean  value.  This  value  gives  an  indication  as  to  whether 
participants  would  use  the  faces  format  in  their  own  work  if  given  the 
opportunity.  A  mean  value  of  3.0  would  indicate  that  half  of  the 
participants  would  probably  use  the  faces  if  given  the  chance. 

Parts  four  and  five  were  not  meant  to  provide  a  statistical  basis  for 
the  usefulness  of  the  faces  format.  They  were  meant  to  ascertain  if 
people  felt  comfortable  with  the  use  of  faces  and  whether  the  might 
actually  use  faces  in  their  work.  Mean  values  of  three  or  greater  in 
these  areas  indicate  that  people  feel  comfortable  enough  with  the  use  of 
faces  that  further  research  into  the  application  of  faces  is  warranted. 
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Faces  Experiment  Survey  Disk  No. 


Use  the  scale  below  to  answer  the  following  questions: 


1  2 


Strongly 

DisAgree 


Strongly 

Agree 


.1.  I  felt  very  confident  answering  the  experiment  questions 
when  the  data  was  presented  in  the  form  of  faces. 

.2.  I  felt  very  confident  answering  the  experiment  questions 
when  the  data  was  presented  in  tabular  form. 

3.  The  faces  format  is  very  accurate. 

4.  The  tabular  format  is  very  accurate. 

5.  The  faces  format  was  easy  to  understand. 


6.  The  tabular  format  was  easy  to  understand. 

7.  I  did  better  answering  questions  (got  more  correct 
answers)  using  faces  data  than  I  did  using  tabular  data. 

8.  If  I  had  the  opportunity  I  would  use  the  faces  format 
in  my  job. 


Figure  4  -  Faces  Experiment  Survey 
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Test  for  Impact  of  Experience  Levels.  The  tests  for  decision  making 
effectiveness  were  redone  to  determine  the  impact  that  experience  levels 
have  on  the  experimental  results.  In  this  case,  the  experimental  data 
were  divided  based  on  the  experience  levels  (experience  using  cost 
performance  measurement)  of  the  participants.  The  data  were  used  to 
test  hypothesis  six  to  determine  if  the  above  results  hold  at  different 
experience  levels. 

Tests  for  Impact  of  Learning  and  Teaching.  As  previously  mentioned, 
upon  analysis  of  the  experiment  data  from  the  first  thirty  samples,  it 
became  evident  that  learning  was  occurring.  As  the  experiment 
progressed  (from  screen  one  to  screen  eight)  participants  were  answering 
questions  faster  (although  with  similar  accuracy) .  Hypothesis  eight 
was  used  to  determine  if  learning  did  occur.  The  experiment  data  was 
divided  into  halves,  with  the  data  from  the  first  half  compared  against 
data  from  the  second  half.  Based  on  these  initial  results,  the 
investigator  modified  the  experiment  by  giving  the  participants  a  five 
minute  lecture  in  the  use  of  faces,  as  well  as  modifying  the  handout 
with  the  sample  faces  by  including  an  explanation  of  how  each 
variable/facial  characteristic  pair  operates.  The  investigator  then 
tested  hypotheses  eight  through  twelve. 

Statistical  Analysis 

All  hypotheses  were  tested  using  either  a  paired  t  test  or  the 
Wilcoxon  Sign-Rank  Test.  All  tests  were  done  using  the  STATISTIX 
SOFTWARE  PACKAGE  (12) .  The  type  of  test  used  depended  upon  the 
distribution  of  the  data  to  be  tested.  A  Wilk-Shapiro  test  for 
normality  was  used  to  determine  whether  the  data  were  normally 
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distributed.  If  the  test  indicated  a  normal  distribution,  the 
parametric  paired  t— test  was  used .  If  the  data  failed  the  test  for 
normality,  the  distribution  free,  Wilcoxon  Sign-Rank  test  was  used, 
tests  were  conducted  using  a  level  of  significance  of  0.05.  (4:290- 

307,  598-607) 


All 
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IV.  Analysis  and  Results 


Overview 

The  purpose  of  this  study  is  to  determine  whether  the  faces  method  of 
representing  cost  performance  information  is  a  viable  tool  for  analyzing 
such  data.  The  main  concern  of  the  study  is:  Does  the  use  of  faces 
result  in  effective  decision  making?  Effectiveness  is  defined  in  terms 
of  decision  accuracy  and  speed.  Confidence  in  making  the  decision  is 
also  considered  a  critical  factor. 

In  order  to  analyze  the  effectiveness  of  the  faces  method,  an 
experiment  was  performed  comparing  the  faces  method  to  a  common  form  of 
cost  performance  data  presentation,  tables.  Several  research  hypotheses 
were  formed  to  measure  the  effectiveness  of  the  faces  method.  Ihese 
research  hypotheses  were  tested  by  means  of  experimentation  and  survey. 
This  chapter  presents  an  analysis  of  the  results  that  were  obtained. 

Several  additional  hypotheses  were  formed  during  the  operation  of  the 
experiment.  The  premise  of  these  hypotheses  is  that  both  learning  and 
training  play  an  important  role  and  that  both  will  result  in  more 
effective  and  more  confident  decisions.  These  hypotheses  were  also 
tested  and  the  analysis  of  those  test  results  are  also  included  in  this 
chapter . 

In  order  to  be  consistent  with  the  format  used  in  the  statistical 
tests,  each  hypothesis  has  been  restated  in  terms  of  a  null  and 
alternate  hypothesis. 
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Population 


Fifty  individuals,  48  military  ana  2  civilians,  participated  in  the 
experiments  of  this  research  study.  Thirty  individuals,  all  military, 
participated  in  the  first  part  of  the  study.  The  remaining  twenty 
participated  in  the  second  part.  The  average  individual  was  a  an  Air 
Force  captain,  with  a  2724  Air  Force  Specialty  Code  (AFSC) ,  having  four 
years  experience  with  the  2724  AFSC,  and  less  than  one  year  of  CXSCSC 
experience.  Table  3  shows  breakdown  of  experiment  participants  by  AFSC, 
rank,  years  of  experience  with  current  AFSC  and  with  using  C/SCSC. 

The  population  statistics  show  that  most  of  the  individuals  involved 
in  the  experiments  have  been  involved  in  acquisition  related  activities 
in  the  Air  Force.  These  individuals  are  involved  in  acquisition  related 
graduate  programs  and  they  will  eventually  have  the  responsibility  of 
evaluating  contractor  cost  performance  information  in  the  jobs.  These 
qualifications  make  them  excellent  subjects  for  evaluating  methods  of 
cost  performance  data  presentation,  especially  those  methods  that  can  be 
used  to  make  decisions  about  contractor  performance. 

Analysis  Overview 

Effectiveness  in  decision  making  is  defined  in  this  study  as  a 
function  of  both  decision  accuracy  and  decision  speed.  In  this 
analysis,  both  decision  accuracy  and  decision  speed  were  first  evaluated 
separately,  and  then  evaluated  as  a  measure  of  decision  making 
effectiveness.  Another  important  factor,  decision  making  confidence, 
was  also  evaluated.  Decision  makers  place  their  reputations  at  risk. 
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TABLE  3 


Population  Statistics 


COUNT 


RANK 

Experiment  1 

Experiment  2 

Total 

First  Lieutenant 

0  /  0.0% 

5  /  25.0% 

5  /  10.0% 

Captain 

28  /  93.3% 

13  /  65.0% 

41  /  82.0% 

Major 

2  /  6.7% 

0  /  0.0% 

2  /  4.0% 

GS-12 

0  /  0.0% 

1  /  5.0% 

1  /  2.0% 

GM-13 

0  /  0.0% 

1  /  5.0% 

1  /  2.0% 

Total 

30  /100.0% 

20  /100.0% 

50  /100.0% 

AFSC 

27XX 

21  /  70.0% 

9  /  45.0% 

30  /  60.0% 

28XX 

5  /  16.7% 

2  /  10.0% 

7  /  14.0% 

65XX 

1  /  3.3% 

1  /  5.0% 

2  /  4.0% 

67XX 

3  /  10.0% 

6  7  30.0% 

9  /  18.0% 

1102 

0  /  0.0% 

1  /  5.0% 

1  7  2.0% 

1515 

0  /  0.0% 

1  /  5.0% 

1  /  2.0% 

Total 

30  /100.0% 

20  /100.0% 

5U  /100.0% 

Years  Experience 

in  AFSC 

<  2.0 

4  /  13.3% 

2  /  10.0% 

6  /  12.0% 

2.0  -  4.0 

5  /  16.7% 

8  /  40.0% 

13  /  26.0% 

4.1  -  6.0 

20  /  66.7% 

10  /  50.0% 

30  /  60.0% 

>  6.0 

1  /  3.3% 

0  /  0.0% 

1  /  2.0% 

Total 

30  /100.0% 

20  /100.0% 

50  /100.0% 

Years  C/SCSC  Experience 

None 

19  /  63.3% 

12  /  60.0% 

31  /  62.0% 

<  1.0 

2  /  6.7% 

1  /  5.0% 

3  /  6.0% 

1.0  -  2.0 

5  /  16.6% 

5  /  25.0% 

10  /  20.0% 

2.1  -  4.0 

2  /  6.7% 

2  /  10.0% 

4  /  8.0% 

>  4.0 

2  /  6.7% 

0  /  0.0% 

2  /  4.0% 

Total 

30  7100.0% 

20  7100.0% 

50  7100.0% 
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and  sometimes  their  jobs,  whenever  they  make  a  decision.  It  is 
important  that  they  are  confident  with  the  decisions  that  they  make.  It 
is  also  important  to  the  United  States  Air  Force,  because  of  the  great 
amount  of  money  involved  in  each  contract,  that  each  of  its  decision 
makers  is  making  confident  decisions.  Decision  making  confidence  will 
be  evaluated  as  a  function  of  the  decision  maker's  perceived  confidence, 
the  decision  maker's  perception  of  the  decision  making  tool  as 
being  accurate,  and  the  decision  maker's  ability  to  understand  the 
information  used  to  make  the  decision. 

Finally,  in  order  for  a  decision  making  tool  to  be  worthwhile, 
individuals  must  be  willing  to  use  it.  This  analysis  will  examine 
whether  individuals  will  utilize  the  faces  format  in  their  decision 
making  if  given  the  opportunity. 

Test  for  Decision  Speed  -  Hypothesis  One 

NULL  HYPOTHESIS:  Individuals  will  take  the  same  amount  of  time 

evaluating  data  in  the  faces  format  as  they  will  evaluating 
data  in  the  tabular  format. 

ALTERNATE  HYPOTHESIS:  Individuals  will  take  less  time  evaluating 

data  in  the  faces  format  than  they  will  evaluating  data  in  the 
tabular  format. 

The  experiment  computer  program  enabled  each  participant  to  be  timed 
for  each  question  set.  One  question  set  was  included  for  each  of  the 
eight  data  sets  that  the  individual  encountered  during  the  experiment. 
Four  of  the  data  sets  involved  data  presented  in  the  form  of  faces;  the 
other  four  data  sets  involved  data  presented  in  tabular  form.  The  time 
for  each  question  set  was  recorded  in  seconds.  The  mean  time  required 
to  answer  questions  using  the  faces  format  was  369  seconds.  The  mean 
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time  required  to  answer  questions  using  the  faces  format  was  only  210 
seconds.  This  information  is  summarized  in  Table  4. 

A  one-sided  paired  t  test  was  conducted  to  test  the  hypotheses.  The 
test  was  performed  using  the  STATISTIX  software  package.  Both 
populations  were  normally  distributed  and  it  is  assumed  that  the 
population  variances  are  similar.  To  test  for  normality,  the  Wilk- 
Shapiro  test  for  normality  was  used.  The  Wilk-Shapiro  test  for 
normality  was  used  on  all  data  distributions  in  this  study.  The 
difference  in  decision  speed  was  not  sufficient  at  the  0.05  significance 
level  to  reject  the  null  hypothesis.  The  results  of  the  test  are 
summarized  in  Table  5.  The  results  indicate  that  the  faces  format  did 
not  contribute  to  faster  decisions.  In  fact,  it  is  clear  that  tables 
took  significantly  less  time. 


TABLE  4 

Time  to  Answer  Questions 


(Time  in  seconds) 


MODE 

MEAN 

STANDARD 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

Faces 

368.7 

111.0 

30 

138.0 

566.0 

Tables 

209.7 

60.10 

30 

110.0 

422.0 
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TABLE  5 


Paired  t  Test  -  Time  by  Presentation  Mode 


PAIRED  T  TEOT  FOR  Face  Times  -  Table  Times 

MEAN  158.9  STD  ERROR  17.49 

T  9.09 

DF  29 

P  0.0000 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE:  REJECT  THE  NULL  HYPOTHESIS  IF  T  <  1.699 


Test  for  Decision  Accuracy  -  Hypothesis  Two 

NULL  HYPOTHESIS:  Individuals  will  be  as  accurate  (in  terms  of 

correct  answers)  evaluating  data  in  the  faces  format  as  they 
will  evaluating  data  in  the  tabular  format. 

ALTERNATE  HYPOTHESIS:  Individuals  will  be  more  accurate  evaluating 
data  in  the  faces  format  than  they  will  evaluating  data  in  the 
tabular  format. 

Individuals  were  asked  a  total  of  24  questions  in  the  experiment. 

Half  of  the  questions  involved  data  presented  in  the  form  of  faces  and 
the  other  half  involved  data  presented  in  tabular  form.  The  mean 
number  of  correct  answers  for  face  format  questions  was  7.7  and  the  mean 
number  of  correct  answers  for  tabular  format  questions  was  10.17.  The 
data  are  summarized  in  Table  6. 

To  test  the  hypotheses,  a  one-sided  paired  t  test  was  used.  The 
results  of  the  test  are  shown  in  Table  7.  The  difference  in  correct 
answers  achieved  for  each  mode  was  not  sufficient  at  the  0.05 
significance  level.  The  null  hypothesis  was  not  rejected.  The  results 
indicate  that  the  use  of  faces  did  not  result  in  more  accurate 
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decisions.  Again,  it  is  clear  that  more  correct  answers  resulted  from 
tabular  data. 


TABLE  6 

Correct  Answers 


(Out  of  12) 


MODE 

MEAN 

STANDARD 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

FACES 

7.667 

1.647 

30 

4.000 

11.00 

TABLES 

10.170 

9.129E-01 

30 

8.000 

12.00 

TABLE  7 

Paired  t  Test  -  Correct  Answers  by  Presentation  Mode 


PAIRED  T  TEST  FOR  Face  Answers  -  Table  Answers 

MEAN  -2.500  SID  ERROR  3.482E-01 

T  -7.18 

DF  29 

P  0.0000 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE  -  REJECT  THE  NULL  HYPOTHESIS  IF  T  >  1.699 
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Test  for  Decision  Making  Effectiveness  -  Hypothesis  Three 


NULL  HYPOTHESIS:  Individuals  will  be  as  effective  making  decisions 
when  evaluating  data  in  the  faces  format  as  they  will 
evaluating  data  in  the  tabular  format. 

ALTERNATE  HYPOTHESIS:  Individuals  will  be  more  effective  making 

decisions  when  evaluating  data  in  the  faces  format  than  they 
will  evaluating  data  in  the  tabular  format. 

For  this  study,  decision  effectiveness  has  been  defined  as  a  function 
of  both  decision  speed  and  decision  accuracy.  Decision  effectiveness  is 
equal  to  decision  accuracy  divided  by  decision  speed  (in  hours). 

The  mean  value  for  decision  effectiveness  when  the  faces  format  is 
used  is  84.05  correct  answers  per  hour.  The  mean  value  for  decision 
effectiveness  when  the  tabular  format  is  used  is  188.10  correct  answers 
per  hour.  The  results  are  summarized  in  Table  8. 

The  paired  t  test  could  not  be  used  in  this  case  because  the  face 
format  decision  effectiveness  data  distribution  failed  the  Wilk-Shapiro 
test  for  normality.  The  hypotheses  were  tested  with  the  Wilcoxon 
Signed-Rank  Test.  Again,  a  one-sided  test  was  used.  The  difference  was 
not  significant  at  the  0.05  significance  level  to  reject  the  null 
hypothesis.  The  results  indicate  that  the  use  of  faces  do  not  lead  to 
more  effective  decisions.  The  results  of  the  Wilcoxon  Signed-Rank  Test 
are  shown  in  Table  9. 
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TABLE  8 


Decision  Effectiveness 


STANDARD 


MODE 

MEAN 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

FACES 

84.05 

39.24 

30 

31.65 

216.4 

TABLES 

188.10 

57.37 

30 

85.31 

360.0 

TABLE  9 

Sign-Rank  Test  -  Decision  Effectiveness 


WILCOXON'S  SIGNED  RANK  TEST  FOR  Face  Effectiveness  -  Table  Effectiveness 


SUM  OF  NEGATIVE  RANKS  -465.0 

SUM  OF  POSITIVE  RANKS  0.000 

NORMAL  APPROXIMATION  WITH  CONTINUITY  CORRECTION  -4.772 

TWO  TAILED  P  VALUE  FOR  NORMAL  APPROXIMATION  0.0000 


TOTAL  NUMBER  OF  VALUES  WHICH  WERE  TIED  0 
NUMBER  OF  ZERO  DIFFERENCES  DROPPED  0 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.00E-05 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE:  REJECT  THE  NULL  HYPOTHESIS  IF  NORMAL 
APPROXIMATION  >1.645 
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Test  for  Decision  Making  Effectiveness  in  Trend  Determination  - 


Hypothesis  Four 

NULL  HYPOTHESIS:  The  mode  of  presentation  (faces  or  tables)  will  not 
affect  the  individuals  ability  (decision  making  effectiveness) 
to  detect  trends. 

ALTERNATE  HYPOTHESIS:  When  attempting  to  evaluate  trends  in  data, 

individuals  will  be  more  effective  when  using  the  faces  format 
than  they  will  using  the  tabular  format. 

Graphical  methods  have  been  used  effectively  to  aid  the  decision 
maker  in  the  detection  of  trends  in  data.  A  few  of  the  questions  in  the 
experiment  were  designed  to  measure  the  participants  ability  to  detect 
trends  in  the  cost  performance  data. 

Table  10  shows  the  results  from  the  trend  detection  questions.  Based 
on  the  mean  results,  tables  resulted  in  faster,  more  accurate,  and  more 
effective  decisions.  To  test  the  hypotheses,  the  Wilcoxon  Sign-Rank 
tests  in  STATISTIX  was  used.  The  data  were  not  sufficient  at  the  0.05 
significant  level  to  reject  the  null  hypothesis.  The  results  indicate 
that  faces  do  not  produce  more  effective  decisions  with  respect  to  trend 
determination.  The  result  of  the  Wilcoxon  Sign-Rank  test  is  provided  as 
Table  11. 


Test  for  Decision  Confidence  -  Hypothesis  Five 

NULL  HYPOTHESIS:  Individuals  will  feel  as  confident  using  the  faces 
presentation  mode  as  they  will  using  the  tabular  presentation 
mode. 

ALTERNATE  HYPOTHESIS:  Individuals  will  feel  less  confident  when 
using  faces  than  they  will  when  using  tables. 
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TABLE  10 
Trend  Detection 

A.  Decision  Times  (in  seconds) 

STANDARD 

MODE  MEAN  DEVIATION  COUNT  MINIMUM  MAXIMUM 

FACES  93.73  32.61  30  38.00  192.0 

TABLES  72.77  29.56  30  43.00  196.0 

B.  Decision  Accuracy  (correct  answers  out  of  four) 

FACES  2.000  7.878E-01  30  0.000  3.000 

TABLES  2.300  8.769E-01  30  1.000  4.000 

C.  Decision  Effectiveness 

FACES  87.39  54.77  30  0.00  284.2 

TABLES  127.90  70.60  30  36.73  334.9 

To  test  for  decision  confidence,  participants  were  asked  to  respond 
to  six  statements  dealing  with  their  perceived  confidence  in  answering 
the  experiment  questions,  and  their  confidence  in  the  mode  of 
presentation  (based  on  perceived  ease  of  use  and  accuracy) .  Three 
statements  dealt  with  the  use  of  faces  and  the  other  three  dealt  with 
the  use  of  faces.  Responses  to  the  statements  were  based  on  a  five 
point  Likert  scale,  with  five  being  the  most  confident,  and  one  the 
least  confident.  The  results  of  the  survey  are  shown  in  Table  12. 
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TABLE  11 


Sign-Rank  Test  for  Trend  Detection  Decision  Effectiveness 


WILOOXON'S  SIGNED  RANK  TEST  FOR  Face  Trend  Effectiveness  -  Table  Trend 
Effectiveness 


SUM  OF  NEGATIVE  RANKS  -342.0 

SUM  OF  POSITIVE  RANKS  123.0 

NORMAL  APPROXIMATION  WITH  CONTINUITY  CORRECTION  -2.242 

TWO  TAILED  P  VALUE  FOR  NORMAL  APPROXIMATION  0.0250 


TOTAL  NUMBER  OF  VALUES  WHICH  WERE  TIED  0 
NUMBER  OF  ZERO  DIFFERENCES  DROPPED  0 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.00E-05 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE:  REJECT  THE  NUIX  HYPOTHESIS  IF  NORMAL 
APPROXIMATION  >1.645 


Both  populations,  faces  confidence  level  and  tables  confidence  level, 
are  normally  distributed  based  on  the  Wilk-Shapiro  test  for  normality. 
Therefore,  the  paired  t  test  in  STATISTIX  was  used  to  test  the 
hypotheses.  The  results  of  the  paired  t  test  are  shown  as  Table  13. 

The  difference  in  confidence  levels  is  significant  at  the  0.05  level  and 
the  null  hypothesis  is  rejected.  The  results  indicate  that  the 
participants  felt  less  confident  making  decisions  using  faces  than  they 
did  when  making  decisions  using  tables. 

Test  to  Determine  if  Individuals  Would  Use  Faces  in  Jobs 

To  measure  if  individuals  would  use  the  faces  technique  in  their  jobs 
they  were  asked  to  respond  to  the  statement  "If  I  had  the  opportunity  I 
would  use  the  faces  format  my  job."  The  responses  were  made  on  a  five 
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point  Likert  scale.  A  response  of  five  meant  that  the  individual 
strongly  agreed  with  the  statement  and  a  response  of  1  meant  that  the 
individual  strongly  disagreed  with  the  statement.  Table  14  is  a  summary 
of  the  responses  to  that  statement.  The  response  shows  that  40  percent 
of  the  participants  would  consider  the  use  of  faces  in  their  jobs. 


TABLE  12 

Survey  Responses  to  Decision  Making  Confidence  Statements 


SURVEY  STANDARD 


STATEMENT 

MEAN 

DEVIATION 

N 

COMMENT 

ONE 

2.650 

1.060 

30 

Faces  -  Perceived  Confidence 

TWO 

4.083 

8.313E-01 

30 

Tables  -  Perceived  Confidence 

THREE 

2.500 

1.137 

30 

Faces  -  Perceived  Accuracy 

POUR 

4.533 

7.761E-01 

30 

Tables  -  Perceived  Accuracy 

FIVE 

3.100 

1.348 

30 

Faces  -  Perceived  Ease  of  Use 

SIX 

4.117 

0.962 

30 

Tables  -  Perceived  Ease  of  Use 

FACE  TOTAL 

8.250 

3.059 

30 

Sum  of  ONE,  THREE,  and  FIVE 

TABLE  TOTAL 

12.730 

2.067 

30 

Sum  of  TWO,  POUR,  and  SIX 

50 


TABLE  13 


Decision  Making  Confidence  Vs  Presentation  Mode 


PAIRED  T  TEST  PC*?  Face  Confidence  -  Table  Confidence 

MEAN  -4.483  BID  ERROR  7.851E-01 

T  -5.71 

DF  29 

P  0.0000 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE  -  REJECT  NULL  HYPOTHESIS  IF  T  >  1.699 


TABLE  14 

Willingness  of  Individuals  to  Use  Face  Format 


RESPONSE  TO  FIVE  POINT  LIKERT  SCALE 


VALUE 

N 

PERCENT 

HISTOGRAM 

1.000 

5 

16.67 

l 

i 

]  ******************* 

» 

2.000 

13 

43.33 

i 

|************************************************** 

I 

3.000 

7 

23.33 

t 

|  *************************** 
i 

4.000 

4 

13.33 

i 

] *************** 
l 

5.000 

1 

3.34 

i 

[  **** 

Totals 

STANDARD 

MEAN  DEVIATION  N 
2.417  0.18  30 
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Test  to  Determine  Impact  of  Experience  on  Decision  Effectiveness  - 


Hypothesis  Six 

NULL  HYPOTHESIS:  Individuals  without  experience  in  analyzing  cost 
performance  data  will  score  the  same  on  decision  making 
effectiveness  when  analyzing  face  data  as  those  individuals 
with  cost  performance  report  experience. 

ALTERNATE  HYPOTHESIS:  Individuals  without  experience  in  analyzing 
cost  performance  data  will  not  score  the  same  on  decision 
making  effectiveness  when  analyzing  face  data  as  those 
individuals  with  cost  performance  report  experience. 

During  the  experiment,  participants  were  asked  to  list  the  amount  of 
experience  they  had  in  using  C/SCSC.  This  was  used  as  a  measure  of 
their  experience  in  using  cost  performance  data.  To  test  the  above 
hypothesis  the  scores  for  decision  effectiveness  of  those  having  less 
than  one  year  experience  were  compared  with  the  scores  of  those  having 
more  than  one  year  experience.  A  summary  of  that  data  is  shown  in  Table 
15. 

Because  the  distributions  failed  the  Wilk-Shapiro  test  for  normality, 
the  Wilcoxon  Sign  rank  test  was  used  to  test  the  hypotheses.  The 
results  of  this  test  are  shown  in  Table  16.  The  differences  in  decision 
making  effectiveness  was  not  significant  at  the  0.05  level.  The  null 
hypothesis  is  not  rejected.  The  data  indicate  that  the  level  of  cost 
performance  experience  did  not  have  an  effect  on  decision  making 
effectiveness  when  faces  were  used. 
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TABLE  15 


Comparison  of  Experiment  Data:  Experience  Vs  No  Experience 
A.  Decision  Times  (in  seconds) 


STANDARD 


EXPERIENCE  MEAN 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

NO 

385.3 

98.57 

21 

251.0 

556.0 

YES 

329.8 

134.1 

9 

138.0 

566.0 

B. 

Decision  Accuracy 

(correct  answers 

out  of  12) 

NO 

7.524 

1.750 

21 

4.000 

11.00 

YES 

8.000 

1.414 

9 

7.000 

11.00 

C. 

Decision  Making  Effectiveness 

NO 

75.29 

25.95 

21 

31.65 

126.6 

YES 

104.5 

56.79 

9 

50.88 

216.4 

TABLE  16 

Decision  Making  Effectiveness  for  Faces  Mode  Vs  Experience 
WILOOXON'S  SIGNED  RANK  TEST  FOR  Experience  VS  No  Experience 


SUM  OF  NEGATIVE  RANKS  -15.00 

SUM  OF  POSITIVE  RANKS  30.00 

EXACT  PROBABILITY  OF  A  RESULT  AS  OR  MORE 

EXTREME  THAN  THE  OBSERVED  RANKS  (1  TAILED  P  VALUE)  0.2168 

NORMAL  APPROXIMATION  WITH  CONTINUITY  CORRECTION  0.829 

TWO  TAILED  P  VALUE  FOR  NORMAL  APPROXIMATION  0.4069 


TOTAL  NUMBER  OF  VALUES  WHICH  WERE  TIED  0 
NUMBER  OF  ZERO  DIFFERENCES  DROPPED  0 

MAX.  DIFF.  ALLOWED  BETWEEN  TIES  1.00E-05 

CASES  INCLUDED  9  MISSING  CASES  12 

DECISION  RULE:  REJECT  NULL  HYPOTHESIS  IF  NORMAL  APPROXIMATION  <  -1.96 

OR  NORMAL  APPROXIMATION  >  1.96 
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Impact  Of  Learning  -  Hypothesis  Seven 

NULL  HYPOTHESIS:  During  the  experiment  learning  did  not  occur  and 
decision  making  effectiveness  was  the  same  in  both  halves  of 
the  experiment. 

ALTERNATE  HYPOTHESIS:  During  the  experiment  learning  occurred  in 
such  a  way  that  decision  making  effectiveness  was  greater  in 
the  bottom  half  of  the  experiment. 

During  the  evaluation  of  the  experiment  results,  it  became  evident 
that  learning  had  taken  place  during  the  experiment.  By  dividing  the 
experiment  into  halves,  one  can  see  that  decision  making  effectiveness 
was  much  greater  during  the  second  half.  Since  the  eight  data  sets  used 
in  the  experiment  were  distributed  so  that  each  individual  data  set 
appeared  in  the  top  half  (first  four  data  sets)  as  many  times  as  it 
appeared  in  the  bottom  half  (last  four  data  sets),  this  comparison  could 
be  made.  Table  17  shows  the  comparison  of  decision  making  effectiveness 
for  both  halves  of  the  experiment. 

A  one-sided  paired  t  test  was  used  to  evaluate  the  impact  of  learning 
on  decision  making  effectiveness  when  faces  where  used.  The  results  of 
the  paired  t  test  are  shown  in  Table  18.  The  difference  in  decision 
making  effectiveness  is  significant  at  the  0.05  level.  The  null 
hypothesis  is  rejected.  The  results  indicate  that  learning  occurred 
during  the  experiment,  and  as  a  result,  decision  making  effectiveness 
increased. 

Impact  of  Teaching  on  the  Use  of  Facial  Presentation 

The  second  experiment  that  was  performed  was  exactly  the  same  as  the 
first  with  one  exception.  In  the  second  experiment,  participants  were 
given  five  minutes  of  instruction  how  Lu  analyse  cost  pex  Tor  manee  data 
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TABLE  17 


Decision  Making  Effectiveness:  Summary  of  Faces  Experiment 
Data  for  Both  Halves  of  Experiment 


PRESENTATION  MODE:  FACES 


HALF 

MEAN 

STANDARD 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

FTOPEFF 

66.68 

39.31 

30 

18.18 

197.8 

FBOIEFF 

116.9 

53.45 

30 

22.36 

234.8 

TABLE  18 

Paired  t  Test  for  Decision  Making 
Effectiveness  Vs  Experiment  Half 


PAIRED  T  TEST  FOR  Bottom  Half  efficiency  -  Top  Half  Efficiency 

MEAN  50.20  SID  ERROR  9.542 

T  5.26 

DF  29 

P  0 . 0000 

CASES  INCLUDED  30  MISSING  CASES  0 

DECISION  RULE:  REJECT  THE  NULL  HYPOTHESIS  IF  T  >  1.699 


using  the  faces  data  presentation  data.  This  section  provides  a 
comparison  of  experimental  results  between  the  second  and  first 
experiments.  The  comparisons  are  limited  to  cases  involving  the  faces 
data  presentation  mode  only.  This  limitation  is  used  since  the  only 
difference  in  experiment  construction  involves  the  faces.  (Further 
analysis  has  shown  that  the  change  did  not  have  any  significant  impact. 
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at  the  0.05  level,  on  decision  making  effectiveness  and  decision  making 
confidence  when  the  tabular  format  was  used.) 

Impact  of  Teaching  on  Decision  Making  Effectiveness 

To  test  for  the  impact  of  teaching  on  decision  making  effectiveness, 
effectiveness  data  from  both  experiments  were  compared.  Since  decision 
making  effectiveness  is  a  function  of  both  decision  speed  and  decision 
accuracy,  hypotheses  related  to  decision  speed  and  decision  accuracy 
were  tested  as  well  as  a  hypothesis  related  to  decision  making 
effective. .ess.  A  one  sided  paired  t  test  was  used  to  test  each  set  of 
hypotheses.  The  experiment  data  for  each  case  is  summarized  in  Table 
19. 


Eecision  Speed  -  Hypothesis  Eight. 

NULL  HYPOTHESIS:  Individuals  who  received  the  five  minute 
instruction  on  the  use  of  faces  will  answer  the  faces 
related  questions  as  fast  as  those  who  did  not  receive  the 
instruction. 

ALTERNATE  HYPOTHESIS:  Individuals  who  received  the  five  minute 
instruction  will  answer  the  faces  questions  faster  than 
those  who  did  not  receive  the  instruction. 

The  one-sight  paired  t  test  for  this  case  is  shown  in  Table  20.  The 
difference  in  decision  times  is  significant  at  the  0.05  level.  The 
results  indicate  that  the  application  of  the  five  minute  teaching 


resulted  in  faster  decisions. 


Decision  Accuracy  -  Hypothesis  Nine. 


NULL  HYPOTHESIS:  Individuals  who  received  the  five  minute 

instruction  on  the  use  of  faces  will  answer  the  faces  related 
questions  as  accurately  (based  on  number  of  correct  answers)  as 
those  who  did  not  receive  the  instruction. 

ALTERNATE  HYPOTHESIS:  Individuals  who  received  the  five  minute 
instruction  will  answer  the  faces  questions  more  accurately  than 
those  who  did  not  receive  the  instruction. 

The  one  sided  paired  t  test  for  this  case  is  shown  in  Table  21.  The 
difference  in  correct  answers  for  the  two  conditions  is  not  significant 
at  the  0.05  level.  The  results  indicate  that  the  use  of  the  five  minute 
instruction  did  not  result  in  more  accurate  decisions. 


TABLE  19 

Comparison  of  Experiment  Data: 

5  Minute  Teaching  Vs.  No  Teaching 


A.  Decision  Times  (in 

TEACHING?  MEAN 

seconds) 

STANDARD 

DEVIATION 

COUNT 

MINIMUM 

MAXIMUM 

NO 

368.7 

111.0 

30 

138.0 

566.0 

YES 

323.0 

77.36 

20 

206.0 

475.0 

B. 

Decision  Accuracy 

(correct  answers  out 

of  twelve) 

NO 

7.667 

1.647 

30 

4.000 

11.00 

YES 

7.800 

1.824 

20 

4.000 

11.00 

C. 

Decision  Effectiveness 

NO 

84.05 

39.24 

30 

31.65 

216.4 

YES 

91.31 

31.52 

20 

56.69 

180.8 
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TABLE  20 


Paired  t  Test  for  Decision  Speed  Vs  Teaching 


PAIRED  T  TEST  TOR  Speed/Teaching  -  Speed/No  Teaching 


MEAN 

-67.50 

STD  ERROR 

27.95 

T 

-2.42 

DF 

19 

P 

0.0260 

CASES 

INCLUDED  20 

MISSING 

CASES  10 

DECISION  RULE  -  REJECT  NULL  HYPOTHESIS  IF  T  <  -1.729 


TABLE  21 

Paired  t  Test  for  Decision  Accuracy  Vs  Teaching 


PAIRED  T  TEST  FOR  Accuracy/Teaching  -  Accuracy /No  Teaching 


MEAN 

4.500E-01 

STD  ERROR 

5 . 305E-01 

T 

0.85 

DF 

19 

P 

0.4069 

CASES 

INCLUDED  20 

MISSING 

CASES  10 

DECISION  RULE  -  REJECT  NULL  HYPOTHESIS  IF  T  >  1.729 


Decision  Making  Effectiveness  -  Hypothesis  Ten. 

NULL  HYPOTHESIS:  Individuals  who  received  the  five  minute 

instruction  on  the  use  of  faces  will  score  the  same  in  decision 
making  effectiveness  when  answering  faces  related  questions  as 
those  who  did  not  receive  the  instruction. 

ALTERNATE  HYPOTHESIS:  Individuals  who  received  the  five  minute 
instruction  will  score  higher  in  decision  making  effectiveness 
than  those  who  did  not  receive  the  instruction. 
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The  results  of  the  one-sided  paired  t  tests  for  this  case  is  shown  in 
Table  22.  The  difference  in  decision  making  effectiveness  was 
sufficient  at  the  0.05  level  to  reject  the  null  hypothesis.  The  results 
indicate  that  the  use  of  the  five  minute  teaching  session  did  result  in 
more  effective  decision  making. 


TABLE  22 

Paired  t  Test  for  Decision  Making  Effectiveness  vs  Teaching 


PAIRED  T  TEST  FDR  Effect/Teaching  -  Effect/No  Teaching 

MEAN  18.56  STD  ERROR  9.834 

T  1.69 

DF  19 

P  0 . 0744 

CASES  INCLUDED  20  MISSING  CASES  10 

DECISION  RULE  -  REJECT  NULL  HYPOTHESIS  IF  T  >  1.729 


The  increase  in  decision  making  effectiveness  can  be  primarily 
attributed  to  the  significant  decrease  in  the  time  taken  to  make  the 
decision.  Although  decision  accuracy  did  increase,  the  increase  was 
not  significant  enough  to  be  a  vital  factor  in  the  increase  to  decision 
making  effectiveness. 

Impact  of  Teaching  on  Decision  Making  Confidence  -  Hypothesis  Eleven. 

NULL  HYPOTHESIS:  Individuals  who  received  the  teaching  will  be  as 
confident  when  making  decisions  using  faces  as  those 
individuals  who  did  not  receive  the  teaching. 
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ALTERNATE  HYPOTHESIS:  Individuals  who  received  the  teaching  will 
be  more  confident  making  decisions  using  faces  than  those  who 
did  not  receive  the  teaching. 

To  test  for  the  impact  on  decision  making  confidence  that  occurs  when 
experiment  participants  are  given  a  five  minute  teaching  on  how  to  use 
the  faces  to  analyze  cost  performance,  the  results  of  the  responses  to 
the  survey  statements  dealing  with  decision  making  confidence  from  both 
experiments  were  compared.  Again,  only  those  responses  dealing  with 
faces  were  compared.  Table  23  shows  a  summary  of  survey  results.  A 
one-sided  paired  t  test  was  used  to  test  for  the  impact  of  the  teaching 
on  decision  making  confidence.  The  results  of  the  t  test  are  shown  in 
Table  24.  The  difference  in  decision  making  confidence  is  sufficient 
enough  at  the  0.05  significance  level  to  reject  the  null  hypothesis. 

The  results  indicate  that  those  individuals  who  received  a  five  minute 
teaching  concerning  the  use  of  the  faces  data  presentation  technique 
were  more  confident  when  making  decisions  based  on  data  presented  as 
faces  than  those  who  did  not  receive  the  teaching. 

Impact  of  Teaching  on  Willingness  of  Individuals  to  Use  the  Faces 
Format  -  Hypothesis  Twelve 

NULL  HYPOTHESIS:  Individuals  who  received  the  five  minute 

instruction  on  the  use  of  faces  will  be  as  likely  to  use  the 
faces  format  in  their  jobs,  if  given  the  opportunity,  as  those 
who  did  not  receive  the  instruction. 

ALTERNATE  HYPOTHESIS:  Individuals  who  received  the  five  minute 
instruction  on  the  use  of  faces  will  more  likely  to  use  the 
faces  format  in  their  jobs,  if  given  the  opportunity,  than 
those  who  did  not  receive  the  instruction. 

To  test  for  the  impact  of  the  five  minute  teaching  session  on  the  use 
of  faces  on  the  individuals  willingness  to  use  the  faces  in  there  job, 
the  responses  to  statement  eight  ("If  I  had  the  opportunity  I  would  use 
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the  faces  format  in  my  job")  of  the  survey  was  compared  between  both 
experiments.  Responses  were  given  on  a  five  point  likert  scale  with  a 
score  of  "5"  indicating  that  the  individual  strongly  agreed  with  the 
statement  and  a  score  of  "1"  indicating  that  the  individual  strongly 
disagreed  with  the  statement.  A  summary  of  responses  to  this  statement 


TABLE  23 

Survey  Responses  to  Decision  Making  Confidence  Statements 


SURVEY  STANDARD 


STATEMENT 

TEACHING 

MEAN 

DEVIATION 

COUNT 

COMMENT 

ONE 

N 

2.650 

1.060 

30 

Perceived  Confidence 

ONE 

Y 

3.300 

0.979 

20 

Perceived  Confidence 

THREE 

N 

2.500 

1.137 

30 

Perceived  Accuracy 

THREE 

Y 

3.100 

8 . 522E-01 

20 

Perceived  Accuracy 

FIVE 

N 

3.100 

1.348 

30 

Perceived  Ease  of  Use 

FIVE 

Y 

3.700 

1.129 

20 

Perceived  Ease  of  Use 

TOTAL 

N 

8.250 

3.059 

30 

Total  -  No  Teaching 

TOTAL 

Y 

10.10 

2.573 

20 

Total  -  Teaching 
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TABLE  24 


Paired  t  Test  for  Decision  Making  Confidence  Vs.  Teaching 


PAIRED  T  TEST  PCS?  Conf/Teaching  -  Conf/No  Teaching 

MEAN  1.900  SID  ERROR  8.912E-01 

T  2.13 

DF  19 

P  0.0463 

CASES  INCLUDED  20  MISSING  CASES  10 
DECISION  RULE  -  REJECT  NULL  IF  T  >  1.729 


for  both  experiments  is  shown  in  Table  25.  A  one-sided  (right-tailed) 
paired  t  test  was  used  to  test  the  hypotheses.  Results  of  the  test  are 
shown  in  Table  26.  The  difference  in  responses  was  not  significant  at 
the  0.05  level.  The  results  indicate  that  sufficient  data  do  not  exist 
to  conclude  that  those  who  receive  instruction  on  the  use  of  the  faces 
format  are  more  likely  to  use  the  faces  format  in  their  jobs.  Table  27 
shows  a  breakdown  of  the  second  experiment's  survey  data  for  question 
eight.  The  data  show  that  sixty  percent  of  the  participants  who 
received  the  teaching  would  consider  using  the  faces  format  in  their 
job. 
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TABLE  25 

Survey  Responses  to  Survey  Statement  #  8 


TEACHING 

MEAN 

STANDARD 

DEVIATION 

COUNT 

MEDIAN 

MINIMUM 

NO 

2.417 

1.018 

30 

1.000 

5.000 

YES 

3.000 

1.338 

20 

1.000 

5.000 

TABLE  26 

Paired  t  Test  for  Willingness  to  Use  Faces  Vs  Teaching 


PAIRED  T  TEST  FOR  Eight/Teaching  -  Eight/No  Teaching 

MEAN  5 . 500E-01  STD  ERROR  3.662E-01 
T  1.50 

DF  19 

P  0 . 1495 

CASES  INCLUDED  20  MISSING  CASES  10 

DECISION  RULE:  REJECT  THE  NULL  HYPOTHESIS  IF  T  >  1.699 
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TABLE  27 


Willingness  of  Individuals  to  Use  Face  Format 
after  Receiving  Teaching  on  the  Use  of  Faces 


RESPONSE  TO  FIVE  POINT  LIKERT  SCALE 
VALUE  COUNT  PERCENT  HISTOGRAM 


1.000 

3 

15 

l 

i 

j ****************************** 

i 

2.000 

5 

25 

• 

J  ************************************************** 

t 

3.000 

4 

20 

i 

j  **************************************** 

l 

4.000 

5 

25 

i 

j ************************************************** 

l 

5.000 

3 

15 

i 

j  a***************************** 

i 

i 
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V.  Conclusions  and  Recommendations 


In  Chapter  1,  four  investigative  questions  were  set  forth  as  the 
objective  of  this  research  study.  Each  question  is  restated  in  this 
chapter  with  a  discussion  of  whether  that  objective  was  achieved,  and  if 
so,  how  it  was  achieved. 

Investigative  Question  1 

How  can  the  method  of  facial  representation  be  PC  based? 

Although  several  “faces"  programs  have  been  written,  none  have  been 
written  for  use  on  a  PC.  Although  some  of  those  programs  could  have 
been  modified  by  the  investigator  for  use  on  a  PC,  the  investigator 
chose  to  develop  his  own.  This  program  is  shown  in  Appendix  A.  It  is 
written  specifically  for  the  analysis  of  CPR  data.  The  program  was 
written  using  QjickBasic  and  has  been  developed  as  a  "stand  alone" 
program.  That  is,  the  program  has  been  converted  so  that  the  user  does 
not  need  CMickBasic  to  run  it.  The  program  does  require  that  the 
computer  utilize  the  MS  DOS  operating  system.  Modifications  to  the 
program  must  be  made  within  the  QuickBasic  environment. 

The  CPR  FACES  program  can  be  contained  on  a  standard  360K  floppy 
disk.  This  makes  it  easily  transferable  from  user  to  user.  It  can  be 
operated  on  any  IEM  compatible  computer  with  ESA  graphics.  The 
potential  aviete  for  widespread  use  of  the  program  among  Air  Force 
program  personnel. 
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Investigative  Qiestion  2 


What  CPR  variables  should  be  selected  for  use  in  the  CPR  FACES 
program  and  which  facial  characteristics  should  be  tied  to  each 
variable? 

Air  Force  contractor  cost  performance  report  data  are  primarily 
described  by  five  variables.  These  variables  are:  Budgeted  Cost  of 
Work  Performed  (BCWP),  Budgeted  Cost  of  Vrk  Scheduled  (BCWS) ,  Actual 
Cost  of  Work  Performed  (ACWP) ,  Budget  at  Completion  (BAC) ,  and  Estimate 
at  Completion  (EAC) .  From  these  five  variables,  one  can  calculate 
several  measures  of  performance  including  Cost  Variance  (CV) ,  Schedule 
Variance  (SV) ,  Cost  Performance  Index  (CPI),  Schedule  Performance  Index 
(SPI),  and  To  Complete  Performance  Index  (TCPI).  Because  they  are  the 
fundamental  blocks  of  CPR  data,  four  variables  (BCWP,  BCWS,  ACWP,  and 
BAC)  were  chosen  as  inputs  to  the  CPR  FACES  program.  (The  EAC  can  be 
computed  from  BCWP,  ACWP,  and  BAC.)  The  CPR  FACES  program  does  not 
match  those  four  variables  with  facial  characteristics.  Instead,  it 
uses  them  to  calculate  the  five  measures  of  performance  mentioned  above. 
The  actual  pairings  are  described  in  Appendix  A.  The  pairings  reflect 
the  preference  of  the  investigator.  A  study  on  how  different  pairings 
may  impact  the  effectiveness  of  the  faces  is  left  as  a  recommendation 
for  further  study.  None  of  the  research  studies  reviewed  in  the 
literature  review  (Chapter  II)  included  a  basis  or  theory  for  assigning 
variables  to  facial  features. 
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Investigative  Question  3 


How  effective  is  facial  representation  in  conveying  cost  performance 
measurement  data? 

This  research  shows  that  an  individual  using  faces  for  the  first  time 
would  probably  have  difficulty  making  decisions  based  on  those  faces. 

The  individual  would  perform  better  using  tabular  data.  Further 
research  revealed  that  an  individual  who  received  training  in  the  use  of 
faces  would  make  more  effective  decisions  than  one  who  received  no 
training.  The  low  decision  effectiveness  scores  can  be  primarily 
attributed  to  the  participant's  unfamiliarity  with  the  concept.  The 
effectiveness  scores  have  been  compared  to  those  achieved  using  tabular 
data.  Individuals  are  much  more  familiar  with  tabular  data.  CPR  data 
are  traditionally  reported  in  tabular  form. 

The  potential  exists  for  facial  representation  to  become  an  effective 
decision  making  tool.  The  two  factors  that  may  facilitate  this  are 
training  and  experience. 

Investigative  Question  4 

How  does  a  person's  level  of  experience  impact  the  usefulness  of  the 
faces  method? 

Individuals  possessing  greater  than  one  year  of  experience  in  cost 
performance  analysis  scored  higher  (though  not  significantly)  in 
decision  making  effectiveness  when  using  faces  than  those  with  less  than 
one  year  experience.  They  also  scored  higher  when  using  tabular  data. 
The  higher  scores  can  be  attributed  to  their  higher  experience  level, 
however,  experience  can  not  be  said  to  have  made  a  significant  impact  on 
the  usefulness  of  the  faces  method. 
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Conclusion 


The  usefulness  of  facial  representation  as  an  analytical  or  decision 
making  tool  in  the  realm  of  cost  performance  measurement  has  been 
subjected  to  two  primary  concerns.  The  first  concern  was  that  it  had  to 
be  PC  based.  This  was  accomplished  through  the  development  of  the  CPR 
FACES  program.  The  second  concern  was  that  the  use  of  faces  would  lead 
to  effective  decision  making.  The  fulfillment  of  this  is  not  as  clear. 
Faces  did  not  lead  to  as  effective  decision  making  as  did  tabular  data. 
However,  training  and  experience  significantly  increase  effectiveness. 
The  degree  of  training  and  experience  needed  to  raise  the  level  of 
effectiveness  to  a  standard  at  least  equal  to  that  of  the  tabular  method 
is  uncertain. 

One  possible  use  of  the  faces  may  be  as  a  companion  to  tabular  data. 
The  CPR  FACES  program  would  serve  well  in  this  capacity.  The  program 
output  displays  the  faces  above  the  corresponding  tabular  data.  The 
user  has  both  tools  available  to  him/her. 

Although  the  research  analysis  has  shown  that  the  use  of  the  faces 
method  resulted  in  lower  decision  making  effectiveness  than  the  use  of 
tabular  data,  it  has  also  shown  that  potential  exists  to  make  it  an 
effective  analytical  and  decision  making  tool.  Further  research  should 
be  done  to  explore  these  areas  of  potential . 

Recommendations 

This  research  examined  the  effectiveness  of  the  faces  method  of 
presenting  CPR  data  as  compared  to  the  tabular  method.  It  did  not 
compare  the  effectiveness  of  using  both  methods  simultaneously  as  a 
decision  making  tool .  Combining  the  attributes  of  the  two  methods  may 
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prove  to  be  an  effective  decision  making  tool.  A  comparison  of  this 
combined  method  with  both  the  faces  method  and  the  tabular  method  is 
encouraged . 

This  research  showed  that  training  in  the  use  of  faces  significantly 
increased  decision  making  effectiveness.  The  training  provided 
consisted  only  of  a  five  minute  lecture  and  a  handout  describing  how  the 
CPR  variables  and  facial  features  were  paired.  This  area  of  study  is 
still  wide  open  for  further  research.  An  investigator  could  compare 
different  degrees  of  training  and  experience  in  the  use  of  faces  to 
determine  what ,  if  any,  is  an  optimum  amount.  Research  is  needed  to 
determine  if  there  is  a  level  of  training  and  experience  that  would  make 
the  faces  method  as  useful  (in  terms  of  decision  making  effectiveness) 
as  the  tabular  method. 

The  CPR  data/facial  feature  pairings  are  the  basis  for  an  additional 
area  of  research.  The  pairings  developed  in  the  CPR  FACES  program  are 
strictly  based  on  personal  preference  of  the  investigator.  It  may  not 
represent  the  optimal  pairings,  if  such  optimal  pairings  exist.  One 
possible  study  could  alter  these  pairings  and  determine  the  impact  on 
decision  making  effectiveness. 

This  research  did  not  take  into  account  the  decision  styles  and 
preferences  of  the  individual  participants.  In  her  research,  Grigware 
(6)  studied  the  effectiveness  of  various  graphical  data  presentation 
methods  as  well  as  the  tabular  method.  As  one  aspect  of  her  research , 
she  used  a  test  to  classify  experiment  participants  as  being  field 
dependent  or  field  independent.  Field  independent  individuals  are 
characterized  as  being  more  analytical  than  field  dependent  individuals. 
The  field  independent  individuals  scored  much  higher  than  the  field 
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dependent  individuals  when  graphs  were  used  (7:10,  39).  Similar 
research  could  be  done  in  the  realm  of  facial  representation  to 
determine  if  the  faces  method  appeals  more  to  specific  decision  making 
styles. 

Research  in  these  areas  are  warranted  in  order  to  determine  whether 
facial  representation  can  be  a  viable  cost  performance  measurement  tool . 
Present  research  indicates  that  the  potential  exists.  The  CPR  FACES 
program,  which  is  inexpensive  to  duplicate  and  easy  to  use,  has  been 
developed  and  should  facilitate  further  research. 
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Appendix  A:  CPR  FACES  Program 


Program  Overview 

The  CPR  FACES  program  was  written  using  QuickBasic  (10) .  The  intent 
of  the  program  is  to  transform  Cost  Performance  Report  (CPR)  data  into 
cartoon  faces  similar  to  those  developed  by  Chemoff  (1).  Unlike  other 
faces  programs  that  have  previously  been  written,  the  CPR  FACES  is 
written  specifically  for  CPR  data.  The  CPR  FACES  program  was  also 
developed  so  that  it  could  easily  be  used  on  a  personal  computer  (PC) . 
Previous  faces  programs  have  been  written  specifically  for  main-frame 
computers.  The  PC  base  gives  the  program  wider  usefulness.  It  makes 
the  program  available  to  all  Air  Force  program  officers. 

Program  Characteristics 

A  primary  feature  of  the  faces  concept  is  the  ability  of  the  faces  to 
show  the  interaction  between  variables.  Previous  programs  have  used 
complex  algorithms  in  order  to  tie  variables  together.  The  CPR  FACES 
program  avoided  the  use  of  such  algorithms  because  of  the  interaction 
already  inherent  with  CPR  variables.  For  example,  two  variables 
portrayed  by  the  CPR  FACES  experiment  are  cost  variance  (CV)  and  to 
complete  performance  index  (TCPI) .  Both  CV  and  TCP I  use  as  an  input  the 
actual  cost  of  work  planned  (ACWP) .  As  ACWP  changes  so  does  CV  and 
TCPI.  The  CPR  FACES  program  reflects  the  changes  to  both 
simultaneously. 

To  draw  the  faces.  BASIC's  graphic  functions  were  employed. 

Conversion  factors  were  developed  and  used  to  convert  the  CPR  data  to 
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inputs  for  BASIC's  graphic's  commands  (such  as  the  CIRCLE  command  where 

the  inputs  are  radius  and  eccentricity) .  The  CPR  variable/facial 

feature  pairings  used  in  the  program  are  as  follows: 

Shape  of  Face  -  TCPI, 

Eyebrow  Position  -  Cost  Variance, 

Pupil  Position  -  Schedule  Variance, 

Curvature  of  the  Mouth  -  Cost  Performance  Index,  and 
Nose  Length  -  Schedule  Performance  Index. 

Program  Operation 

The  program  was  written  so  that  the  user  could  enter  CPR  data  by  the 
use  of  a  data  file,  "a:cpr.dat."  The  user  cam  enter  up  to  eight  cases, 
with  four  cases  being  the  maximum  that  can  be  displayed  on  one  screen. 
The  program  uses  the  following  as  inputs  from  the  user:  BCWS,  BCWP, 

ACWP,  and  BAC.  The  computer  computes  CV,  SV,  CPI,  and  SPI  for  each 
case.  The  faces  are  drawn  and  a  listing  of  BCWP,  BCWS,  ACWP,  SV,  CV, 
CPI,  and  SPI  is  produced. 

The  first  faces  screen  that  is  produced  during  each  operation  is 
designed  to  illustrate  what  a  very  good,  very  bad,  and  neutral  face 
should  look  like.  The  following  two  screens  are  faces  and  data  based  on 
the  user's  inputs.  Once  the  user  becomes  familiar  with  the  CPR  FACES 
output,  he/she  can  delate  the  first  screen  with  very  little  to  the 
program . 

Unfortunately,  the  CPR  FACES  program  does  not  include  an  option  that 
allows  the  user  to  print  the  output.  The  author  used  WordPerfect  5.0's 
"grab"  utility  to  import  the  faces  output  into  a  WordPerfect  5.0  text 
file.  One  can  learn  this  procedure  by  consulting  the  WordPerfect  5.0 
documentation  (14) . 


72 


The  QuickBasic  program  code  is  listed  below  with  appropriate 
comments.  A  sample  program  output  is  shewn  after  the  code  listing. 


CPR  FACES  Program 

1  Main  Program 

DECLARE  SUB  Ifi  () 

DECLARE  SUB  ichart  () 

DECLARE  SUB  Ifa  () 

DECLARE  SUB  chart 2  () 

DECLARE  SUB  intro  ()  'Declare  Subroutines 

DECLARE  SUB  define  () 

DECLARE  SUB  set3faceinfo  () 

DECLARE  SUB  set3faces  () 

DECLARE  SUB  faces  () 

DECLARE  SUB  cprinfo  () 

DECLARE  SUB  Facelnfo  0 
DECLARE  SUB  Table  () 

DECLARE  SUB  chart  () 


'  Define  Variables 

COMMON  SHARED  a.  r,  mr.  er.  ear.  pr.  era,  eara,  b$ 

COMMON  SHARED  bc(),  mc().  nl(),  ef(),  pf() 

COMMON  SHARED  bcwp() ,  bcvs(),  acwpO,  cv(),  cvp(),  svp(),  bac() 
COMMON  3-1ARED  cpi ( ) ,  spi().  pc(),  ps(),  tcpiO 
COMMON  SHARED  yn() ,  yel(),  xl().  x2(),  sv() 

DIM  be (35) ,  mc(35) ,  nl(35),  ef(35).  pf(35) 

DIM  bcwp(35) .  bcws(35) ,  acwp(35) ,  cv(35),  cvp(35) .  svp(35),  bac(35) 
DIM  cpi (35),  spi(35),  pc(35),  ps(35),  tcpi(35) 

DIM  yn(35) ,  yel(35),  xl(35),  x2(35),  sv(35) 

DIM  introtext$(24) 


CLS 

OPEN  "a:cpr.dat"  FOR  INPUT  AS  #2 

a  -  1 
5 

INPUT  #2,  bcws(a),  bcwp(a) ,  acwp(a) ,  bac(a) 

IF  a  -  8  GOTO  6 
a  *  a  +  1 
GOTO  5 
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6 


bcvs(9)  -  2000 
bcwp(9)  -  4000 
acwp{9)  =  2000 
bac(9)  -  15000 
bcws(10)  -  10000 
bcwp(10)  -  4000 
acwp(lO)  -  7000 

bac(10)  -=  15000  1  CPR  Data  for  sample  faces 

bcws(ll)  -  4000 

bcwp(ll)  *=  4000 

acwp(ll)  -  4000 

bac(ll)  =  15000 


Intro 

CLS 

OPEN  "a: input.txt"  FOR  INPUT  AS  #3 
VIEW  PRINT  1  TO  24 
b  =  1 
9 

LINE  INPUT  #3,  introtext$(b) 

PRINT  introt ext$(b) 

b  -  b  +  1 

IF  b  <  22  GOTO  9 

3 

IOCATE  23.  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  =  INKEYS 

IF  hits  <>  ""  THEN  GOTO  4  ELSE  GOIO  3 

4 

CLS  0 


cprinfo 

Ifi  'DRAW  SAMPLE  FACES  AND  TABLE 

If  a 
i  chart 

31 

LOCATE  23,  10 

F’RINT  "Press  Any  Key  to  Continue" 
hits  -  INKEYS 

IF  hits  <>  ""  THEN  GOTO  41  ELSE  GOIO  31 
41 

10 

CLS  0 
Facelnfo 

faces  'DRAW  SCREEN  1  FACES  AND  TABLE 
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20 

chart 

21 

LOCATE  23,  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  -  INKEYS 

IF  hit$  <>  ""  THEN  GOTO  30  PT.qp  GOTO  21 
30 

CLS  0 

set2faceinfo  'CRAW  SCREEN  2  FACES  AND  TABLE 

set2faces 

chart2 

40 

LOCATE  23,  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  =  INKEYS 

IF  hits  <>  ""  THEN  GOTO  90  ELSE  GOTO  40 


90 

CLS  0 

VIEW  PRINT  1  TO  24 
LOCATE  4.  1 

PRINT  "This  completes  the  faces  demo.” 
99 

END 


SUBROUTINES 


SUB  chart  'CPR  TABULAR  DATA  FOR  DATA  SCREEN  1 

VIEW  PRINT  13  TO  24 
LOCATE  15.  1 

PRINT  "  BCWS  BCWP  ACWP  SV  CV  SPI 

CPI" 

LOCATE  17,  1 

PPTKIT  ! RTMH  "  A  jjyyyyfc  ##■####  i i H H ## 

n\Xrii  t/vXlfV  *»  nnniTiTTT  wlnTTnin  n  I!  n  IT  rr  n  wnTTnTrH  nHi!  "»  H  II 

##.*4*  ##.##♦";  bcws(l):  hrwpd);  acwp(l);  sv(l);  cv(  1 ) ;  spi(l); 

cpi(l) 

LOCATE  18,  1 

pDTkTT  n^TMn  "  P  ######  It M # # # 

I  Xv  111  1  UO  X I  iw  XJ  nnnTinn  WnTTirnii  InTinnrW  IriTnnTni 

##.###  ##.###";  bcws(2) ;  bcwp(2) ;  acwp(2) ;  SV(2);  cv(2) ;  spi (2) ; 

cpi (2) 
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LOCATE  19,  1 

prpTMT  IfqTNft  "  C  iHti  M-H  H  jyyt  jyyi  tmyumi  uu  m  u  u  u 

mini  vwinv  v  n  n  ii  ii  ii  u  n  fi  n  ii  •»  «»  innnnnr  innnnnr  TnnrmnT 

##.###  ##.###";  bcws (3) ;  bcwp(3);  acwp(3) ;  SV(3);  CV(3);  spi(3); 

cpi  (3) 

LOCATE  20,  1 

PRINT  USING  "  D  ######  ######  ######  ItiiHlIH  #***#* 

##.###  ##.###";  bcws(4);  bcwp(4) ;  acwp(4);  sv(4);  cv( 4);  spi (4) ; 

cpi (4) 

END  SUB 


SUB  chart  2  'CPR  TABULAR  DATA  FDR  DATA  SCREEN  2 

VIEW  PRINT  13  TO  24 
LOCATE  15,  1 

PRINT  “  BCWS  BCWP  ACWP  SV  CV 

CPI" 

LOCATE  17,  1 

PRINT  USING  "  A  #**###  ####*#  ###*##  ###♦##  #*#### 

##.*##  ##.###";  bcws(5);  bcwp(5);  acwp(5);  sv(5);  cv(5) ;  spi (5) ; 

cpi (5) 

LOCATE  18,  1 

PRINT  USING  "  B  ######  II II H II II II  ######  WiH  ###### 

##.###  ##.###";  bcws(6);  bcwp(6);  acwp(6);  sv(6) ;  cv(6) ;  spi (6); 

cpi (6) 

LOCATE  19,  1 

PRINT  USING  "  C  ######  ######  ######  ######  ####*# 

##.###  ##.###";  bcws(7);  bcwp(7);  acwp(7);  SV(7) ;  cv(7) ;  spi (7) ; 

cpi (7) 

LOCATE  20,  1 

PRINT  USING  "  D  ######  ######  ###*##  #####*  #*#### 

##.##*  ##.###";  bcws(8);  bcwp(8);  acwp(8);  sv(8);  cv(8);  spi (8); 

cpi (8) 

END  SUB 


SUB  cprinfo  ’CALCULATES  SV,  CV,  CPI,  SPI.  AND  TCPI 


6000 
a  -  1 
6100 

IF  a  >  11  THEN  GOTO  6200 

cv(a)  -  bcwp(a)  -  acwp(a) 
sv(a)  -  bcwp(a)  -  bcws(a) 
cvp(a)  -  cv(a)  /  bcwp(a) 
svp(a)  -  sv(a)  /  bcws(a) 

i 

Performance  Indices 


'cost  variance 
'schedule  variance 
'cost  variance  percentage 
'schedule  variance  percentage 


SPI 
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cpi(a)  -  bcwp(a)  /  acwp(a) 

spi(a)  -  bcwp(a)  /  bcws(a) 

» 

pc(a)  =  bcwp(a)  /  bac(a) 
ps(a)  -  acwp(a)  /  bac(a) 

i 

tcpi(a)  =  (bac(a)  -bcwp(a)) 
a  -  a  +  1 
GOTO  6100 


‘Coert  Performance  Index 
‘Schedule  Performance  Index 

'Percent  complete 
’percent  spent 

/  (bac(a)  -  acwp(a)) 


6200 
END  SUB 


SUB  Facelnfo  'CONVERTS  CPR  DATA  TO  FACES  INPUTS 


r  =  25 
mr  -  10 
er  =  7 
ear  =  3 
pr  *=  .75 
era  -  .5 
eara  *=  .8 


'face  radius 
‘mouth  radius 
‘eye  radius 
‘ear  radius 
'pupil  radius 
‘eye  aspect  ratio 
‘ear  aspect  ratio 


a  -  1 


Input  CPR  Data 

i 

100 

IF  a  >  4  GOTO  190 

bc(a)  =  tcpi(a)  *  -.1  +  1 
IF  bc(a)  >  1  THEN  bc(a)  -  1 
IF  be (a)  <  .7  THEN  be (a)  -  .7 

i 

mc(a)  -  cpi (a)  -  1 

IF  me (a)  <  -1  THEN  me (a)  -  -1 

IF  me (a)  >  1  THEN  me (a)  -  1 

i 

nl (a)  -  5  +  spi (a)  *  5 

IF  nl (a)  >  15  THEN  nl(a)  -  15 

i 

ef(a)  -  cvp(a)  *  7 

IF  ef (a)  <  -7  THEN  ef (a)  -  -7 

IF  ef (a)  >  7  THEN  ef(a)  «  7 

i 

pf(a)  -  svp(a)  *  10 
IF  pf (a)  <  -5  THEN  pf(a)  -  -5 
IF  pf (a)  >  5  THEN  pf(a)  -  5 
a  -  a  +  1 
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GOTO  100 
190 

END  SUB 


SUB  faces  ‘DRAWS  FACES  FOR  SCREEN  1 

i 

SCREEN  9 

VIEW  (15,  5)-(615,  125),  8,  4 
WINDOW  (0,  0)-(400,  100) 

a  *=  1 

'  Draw  Face  Outline 

1000 

IF  a  >  4  GOTO  1290 


pi  =  3.14 


xface  =  (a  *  100)  -  50 

CIRCLE  (xface,  50),  r.  .  0,  pi,  1  ‘top  half 
CIRCLE  (xface,  50),  r,  ,  pi,  0,  be (a) 'bottom  half 

i 

'  Draw  mouth  curvature 

IF  me (a)  <=  0  GOTO  1100  ELSE  GOTO  1150 

i 

1100  me (a)  -  -me (a) 

CIRCLE  (xface,  35),  mr,  ,  .2  *  pi,  .8  *  pi,  me (a) 

GOTO  1200 
1150 

CIRCLE  (xface,  40),  mr,  ,  1  *  pi,  0  *  pi,  mc(a) 
1200 

'  eyes 

CIRCLE  (xface  -  10,  60).  er,  ,  ,  ,  era' left  eye 
CIRCLE  (xface  +  10,  60),  er,  ,  .  ,  era'right  eye 

i 

'  riose 

i 

yn(a)  *=  60  -  nl  (a) 

LINE  (xface,  60)-(xface  +  3,  yn(a)) 

LINE  (xface  +  3,  yn(a) )-(xface  -  3,  yn(a)) 

LINE  (xface  -  3,  yn(a) )-(xface,  60) 

i 

ears 
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CIRCLE  (xface  +  29,  50),  ear,  ,  ,  ,  eara 
CIRCLE  (xface  -  29,  50),  ear,  ,  ,  ,  eara 


'  eyebrows 

IF  ef  (a)  >  0  THEN  GOTO  1225 
yel (a)  -  65  +  ef (a) 

LINE  (xface  -  5,  65)-(xface  -  17,  yel (a)) 
LINE  (xface  +  5,  65)-(xface  +  17,  yel (a)) 
GOTO  1250 
1225 

yel (a)  -  65  -  ef(a) 

LINE  (xface  -  17,  65)-(xface  -  5,  yel (a)) 
LINE  (xface  +  17,  65)-(xface  +  5,  yel (a)) 
1250 


'  pupils 

i 

xl(a)  -  (xface  -  10)  +  pf(a) 
x2(a)  *  (xface  +  10)  +  pf(a) 

CIRCLE  (xl (a) ,  60),  pr 
PAINT  (xl(a) ,  60) 

CIRCLE  (x2(a) ,  60),  pr 
PAINT  (x2(a) ,  60) 
a  =  a  +  1 
GOTO  1000 
1290 

VIEW  PRINT  10  TO  24 
LOCATE  11,  10 

PRINT  "A  B  C  D" 

END  SUB 


SUB  i chart  'TABULAR  DATA  FOR  SAMPLE  FACES 

VIEW  PRINT  13  TO  24 
LOCATE  15,  1 

PRINT  "  BCWS  BCWP  ACWP  SV  CV 

CPI" 

LOCATE  17,  1 

PRINT  USING  "GOOD  ######  ######  ######  frtHHMMt  ###### 

##.###  ##.##1";  bcws(9) ;  bcwp(9);  acwp(9) ;  sv(9) ;  cv(9);  spi(9); 

cp’ (9) 

LOCATE  18,  1 
LOCATE  19,  1 

PRINT  USING  "NEUT  ####*#  ######  ######  ######  ###### 

##•###  ##.###";  bcws(ll);  bcwp(ll) ;  acwp(ll);  SV(ll);  CV(ll); 

spi (11) ;  cpi (11) 

END  SUB 


SPI 
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SUB  Ifa 


DRAWS  SAMPLE  FACES 


SCREEN  9 

VIEW  (15.  5)-(615,  125).  8,  4 
WINDOW  (0,  0)-(400,  100) 

a  -  9 

'  Draw  Face  Outline 

31000 

IF  a  >  11  GOTO  31290 
pi  -  3.14 

i 

xface  -  ( (a  -  8)  *  100)  -  50 

CIRCLE  (xface,  50),  r,  ,  0,  pi,  1  'top  half 

CIRCLE  (xface,  50),  r,  ,  pi,  0,  bc(a) 'bottom  half 

i 

'  Draw  mouth  curvature 

i 

IF  me  (a)  <=  0  GOTO  31100  ELSE  GOTO  31150 

l 

31100 

mc(a)  =  -mc(a) 

CIRCLE  (xface,  35) ,  m* ,  ,  .2  *  pi,  .8  *  pi,  me (a) 

GOTO  31200 
31150 

CIRCLE  (xface,  40),  mr,  ,  1  *  pi,  0  *  pi,  mc(a) 
31200 

'  eyes 

i 

CIRCLE  (xface  -  10,  60),  er,  ,  ,  .  era' left  eye 

CIRCLE  (xface  +  10.  60),  er,  ,  ,  ,  era'right  eye 

'  nose 

t 

yn(a)  -  60  -  nl (a) 

LINE  (xface,  60)-(xface  +  3,  yn(a)) 

LINE  (xface  +  3,  yn(a) )-(xface  -  3,  yn(a)) 

LINE  (xface  -  3,  yn(a) )-(xface,  60) 

i 

'  ears 

i 

CIRCLE  (xface  +  29,  50),  ear,  ,  ,  ,  eara 

CIRCLE  (xface  -  29,  50).  ear,  »  ,  ,  eara 

> 

'  eyebrows 

IF  ef (a)  >  0  THEN  GOTO  31225 
yel (a)  -  65  +  ef (a) 

LINE  (xface  -  5,  65)-(xface  -  17,  yel (a)) 
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LINE  (xface  +  5,  65)-(xface  +  17,  yel(a)) 

GOTO  31250 
31225 

yel(a)  =  65  -  ef(a) 

LINE  (xface  -  17,  65)-(xface  -  5,  yel(a)) 

LINE  (xface  +  17,  65)-(xface  +  5,  yel(a)) 

31250 

i 

'  pupils 

i 

xl(a)  ■=  (xface  -  10)  +  pf(a) 
x2(a)  *  (xface  +  10)  +  pf(a) 

CIRCLE  (xl(a),  60),  pr 
PAINT  (xl(a) ,  60) 

CIRCLE  (x2(a),  60),  pr 
PAINT  (x2(a) ,  60) 
a  -  a  +  1 
GOTO  31000 
31290 

LOCATE  11,  10 

PRINT  "  GOOD  BAD  NEUTRAL 

END  SUB 


SUB  Ifi  'CONVERTS  CPR  DATA  TO  FACES  INPUTS  FOR  SAMPLE  FACES 


r  -  25 

'face  radius 

mr  =  10 

'mouth  radius 

er  =  7 

'eye  radius 

ear  -  3 

'ear  radius 

pr  *  .75 

'pupil  radius 

era  *  .5 

'eye  aspect  ratio 

eara  =  . 8 

‘ear  aspect  ratio 

a  -  9 

Input  CPR  Data 

i 

30100 

IF  a  >  11  GOTO  30190 


bc(a)  =  tcpi(a)  *  -.1  +  1 
IF  fcc(a)  >  1  THEN  bc(a)  -  1 
IF  bc(a)  <  .7  THEN  bc(a)  *  .7 

i 


me  (a)  =  cpi(a)  -  1 

IF  me  (a)  <  -1  THEN  me  (a)  -  -1 

IF  mc(a)  >  1  THEN  mc(a)  -  1 
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nl  (a)  *=  5  +  spi(a)  *  5 

IF  nl (a)  >  15  THEM  nl(a)  =  15 

i 

ef(aj  -  cvp(a)  *  7 

IF  ef (a)  <  -7  THEM  ef(a)  -  -7 

IF  ef (a)  >  7  THEM  ef(a)  -  7 

i 

pf(a)  =  svp(a)  *  10 

IF  pf(a)  <  -5  THEM  pf(a)  *=  -5 

IF  pf (a)  >  5  THEM  pf(a)  -  5 

a  *=  a  +  1 

GOTO  30100 

30190 

EMD  SUB 


SUB  intro  '  TITLE  SCREEN 
CLS  0 

VTEW  PRINT  6  TO  24 
LOCATE  7,  25 

PRINT  "Cost  Performance  Report  Faces  Demo" 
LOCATE  8,  35 

PRINT  "by  Capt  Jeffrey  Tkach" 

LOCATE  9,  38 
PRINT  "AFIT/GSM" 

LOCATE  10,  36 

PRINT  "September  1990" 

1 

LOCATE  21,  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  =  INKERS 

IF  hits  <>  ""  THEN  GOTO  2  ELSE  GOTO  1 
2 

CLS 

END  SUB 


SUB  set2faceinfo  '  CONVERTS  CPR  DATA  TO  FACES  INPUTS  FOR  SCREEN  2 


r  =  25 
mr  =  10 
er  -  7 
ear  -  3 
pr  -  .75 
era  -  .5 
eara  =  .8 


'face  radius 
'mouth  radius 
'eye  radius 
'ear  radius 
'pupil  radius 
‘eye  aspect  ratio 
'ear  aspect  ratio 


a 


5 
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Input  CPR  Data 


8100 

IF  a  >  8  GOTO  8190 

i 

bc(a)  =  tcpi(a)  *  -.1  +  1 
IF  be  (a)  >  1  THEM  be  (a)  -  1 
IF  be  (a)  <  .7  THEM  bc(a)  =  .7 

i 

mc(a)  -  cpi(a)  -  1 

IF  me  (a)  <  -1  THEM  me  (a)  -  -1 

IF  me (a)  >  1  THEM  me (a)  -  1 

i 

nl  (a)  -  5  +  spi (a)  *  5 

IF  nl (a)  >  15  THEM  nl (a)  -  15 

i 

ef(a)  *  cvp(a)  *  7 

IF  ef (a)  <  -7  THEM  ef(a)  -  -7 

IF  ef (a)  >  7  THEN  ef(a)  =  7 

i 

pf(a)  «=  svp(a)  *  10 

IF  pf (a)  <  -5  THEM  pf(a  -  -5 

IF  pf (a)  >  5  THEN  pf(a,  =  5 

a  =  a  +  1 

GOTO  8100 

8190 

EMD  SUB 


SUB  set2faces  “DRAWS  FACES  FOR  SCREEM  2 

i 

SCREEM  9 

VIEW  (15.  5)-(615,  125),  8.  4 
WINDOW  (0,  0)-(400,  100) 

a  =  5 


Draw  Face  Oitline 


9000 

IF  a  >  9  GOTO  9290 
> 

pi  =  3.14 

xface  *=  ( (a  -  4)  *  100)  -  50 

CIRCLE  (xface,  50),  r,  ,  0,  pi,  1  'top  half 

CIRCLE  (xface,  50).  r,  ,  pi,  0,  be (a) 'bottom  half 

i 

Draw  mouth  curvature 
IF  me (a)  >  0  GOTO  9150 
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x' 


9100  me (a)  =  -me (a) 

CIRCLE  (xface,  35),  mr,  ,  .2  *  pi,  .8  *  pi,  me (a) 

GOTO  9200 
9150 

CIRCLE  (xface,  40),  mr,  ,  1  *  pi,  0  *  pi,  me (a) 

9200 

‘  eyes 

i 

CIRCLE  (xface  -  10,  60),  er,  ,  ,  ,  era1 left  eye 

CIRCLE  (xface  +  10,  60),  er,  ,  ,  ,  era'right  eye 

i 

'  nose 

t 

yn(a)  -  60  -  ni (aj 

LINE  (xface,  60)-(xface  +  3,  yn(a)) 

LINE  (xface  +  3,  yn(a) )-(xface  -  3,  yn(a)) 

LINE  (xface  -  3,  yn(a) )-(xface,  60) 

l 

'  ears 

CIRCLE  (xface  +  29,  50),  ear,  ,  ,  ,  eara 

CIRCLE  (xface  -  29,  50),  ear,  ,  ,  .  eara 

i 

'  eyebrows 

i 

IF  ef  (a)  >  0  THEN  GOTO  9225 
yel (a)  «  65  +  ef (a) 

LINE  (xface  -  5,  65)-(xface  -  17,  yel (a)) 

LINE  (xface  +  5,  65)-(xface  +  17,  yel (a)) 

GOTO  9250 
9225 

yel  (a)  •=  65  -  ef  (a) 

LINE  (xface  -  17,  65)-(xface  -  5,  yel (a)) 

LINE  (xface  +  17,  65)-(xface  +  5,  yel (a)) 

9250 

i 

'  pupils 

i 

xl(a)  *  (xface  -  10)  +  pf(a) 
x2(a)  =  (xface  +  10)  +  pf(a) 

CIRCLE  (xl (a) ,  60),  pr 
PAINT  (xl (a) ,  60) 

CIRCLE  (x2( a).  60).  pr 
PAINT  (x2(a) ,  60) 
a  =  a  +  1 
GOTO  9000 
9290 

VIEW  PRINT  10  TO  12 
LOCATE  11,  10 

PRINT  "A  B  C  D 

END  SUB 
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CPR  FACES  Sample  Output 


Screen  1  -  Examples 


4 

J e3Ae>\ 

y®A 

voy 

vc/ 

\zJ 

GOOD 

M 

NEUTBAL 

ECUS 

bcup  ftcur 

8M  CM 

SP1 

CPI 

GOOD 

2888 

4888  2888 

2880  2888 

2.088 

2.080 

BAD 

18888 

4088  7888 

*  ***»■>*  «VWVt 

IAWO  JUKI 

8.480 

0.571 

HSUT 

4888 

4888  4888 

8  8 

1.088 

1.888 

Press  Any  Bey  to  Cost i esc 

Screen  2  -  First  Four  Data  Sets 
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Screen  -  Last  Four  Data  Sets 
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Appendix  B:  Experiment  Program 


The  experiment  program  was  written  in  QuickBasic.  It  utilizes  the 
CPR  FACES  program  that  was  shown  in  Appendix  A. 

Two  main  features  characterize  this  program.  The  first  feature  is 
the  ability  to  save  the  participant's  answers  to  the  experiment 
questions  to  floppy  disks.  The  second  feature  is  the  ability  to  record 
the  amount  of  time  that  it  takes  the  participant  to  answer  the 
experiment  questions. 

Below  is  a  partial  listing  of  the  code  used  to  develop  the  experiment 
program.  To  eliminate  redundancy  in  the  text  some  subroutine  codes  have 
been  omitted.  The  complete  program  code  is  available  upon  request. 


Main  Program 


1  Declare  Subroutines 

DECLARE  SUB  If i  () 

DECLARE  SUB  Ichart  () 
DECLARE  SUB  Ifa  () 

DECLARE  SUB  chart3  () 
DECLARE  SUB  set7faceinfo  () 
DECLARE  SUB  set7faces  () 
DECLARE  SUB  chart4  () 
DECLARE  SUB  set5faces  () 
DECLARE  SUB  set 5 face info  0 
DECT  ARE  SUB  quest ions5  () 
DECLARE  SUB  personal  () 
DECLARE  SUB  chart 2  () 
DECLARE  SUB  quest ions4  () 
DECT  ARE  SUB  quest  ions3  () 
DECLARE  SUB  intro  () 
DECLARE  SUB  define  () 
DECLARE  SUB  set3faceinfo  () 
DECLARE  SUB  set3faces  () 
DECLARE  SUB  questions!  (> 
DECLARE  SUB  quest ions2  0 
DECLARE  SUB  questions  () 
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DECLARE  SUB  Faces  () 

DECLARE  SUB  cprinfo  () 

DECLARE  SUB  Facelnfo  () 

DECLARE  SUB  Table  () 

DECLARE  SUB  chart  () 

DECLARE  SUB  faceinfo2  () 

DECLARE  SUB  faces 2  () 

1  Declare  Variables 

COMMON  SHARED  a.  r,  mr,  er,  ear,  pr,  era.  ear  a,  b$ 

COMMON  SHARED  bc(),  mc(),  nl(),  ef(),  pf() 

COMMON  SHARED  bcwp  () ,  bcws(),  acwpO,  cv(),  cvp(),  svp(),  bac() 
COMMON  SHARED  cpi(),  spi(),  pc(),  ps(),  tcpi() 

COMMON  SHARED  yn(),  yel().  xl().  x2(),  sv() 

DIM  be (35) ,  me (35) ,  nl(35),  ef(35),  pf(35) 

DIMbcwp(35),  bcws(35),  acwp(35) ,  cv(35) ,  cvp(35),  svp(35) ,  bac(35) 
DIM  cpi(35),  spi(35),  pc (35).  ps(35),  tcpi(35) 

DIM  yn(35) ,  yel(35).  xl(35),  x2(35).  sv(35) 

DIM  introtext$(24) 

'  Input  Cost  Performance  Data  from  File  "a:facedata.pm" 


OPEN  "a :  f acedata .  pm"  FOR  INPUT  AS  #2 

a  -  1 

5 

INPUT  #2,  bcws(a),  bcwp (a) .  acwp(a),  bac(a) 

IF  a  -  35  GOTO  6 
a  *=  a  +  1 
GOTO  5 

6 


'  Create  data  file  "a:data,dat"  for  experiment  responses. 

OPEN  "a: data.dat"  FOR  OUTPUT  AS  *1 

intro 

CLS 

’  Open  test  file  "a: input .txt"  for  experiment  introduction. 

OPEN  "a : input . txt"  FOR  INPUT  AS  #3 
VIEW  PRINT  1  TO  24 
b  -  1 
9 

LINE  INPUT  #3,  introtext$(b) 

PRINT  introtext$(b) 
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b  -  b  +  1 

IF  b  <  22  GOTO  9 


3 

LOCATE  23,  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  -  INKERS 

IF  hit$  <>  THEN  GOTO  4  ELSE  GOTO  3 

4 


'  Call  subroutine  "personal"  to  collect  background  data  on 
'  participant, 

personal 

'  DRAW  EXAMPLE  FACES  AND  CORRESPONDING  CPR  DATA 

CLS  0 
cprinfo 
If  i 
If  a 

I chart 
31 

LOCATE  23,  10 

PRINT  "Press  Any  Key  to  Continue" 
hit$  *  INKEYS 

IF  hits  <>  ""  THEN  GOTO  41  ELSE  GCTO  31 
41 

‘  EXPERIMENT  DATA  SET  1  -  FACES 

Facelnfo  '  Calls  subroutine  to  convert  CPR  data  to  face  data 
10 

Faces  ’  Calls  subroutine  to  draw  faces 
WRITE  #1,  "SET  1" 

quest ions 1  '  Calls  subroutine  to  display  question  set  1 

'  EXPERIMENT  DATA  SET  2  -  TABLES 


1  Calls  subroutine  to  calculate  CPR  information 
'  Calls  subroutine  to  convert  CPR  data  to  Face  data 
*  Calls  subroutine  to  draw  faces 


20 

CLS  0 

chart  '  Calls  subroutine  to  present  data  in  table  form 
WPTTE  #1,  "SET  2" 

quest ionsl  '  Calls  subroutine  to  display  questions 
30  1  EXPERIMENT  DATA  SET  3  -  FACES 

CLS  0 
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set3faceinfo 
set3faces 
WRITE  #1,  "SET  3" 
quest ions2 

40  1  EXPERIMENT  DATA  SET  4  -  TABLES 

CLS  0 
chart 2 

WRITE  #1,  "SET  4" 
quest ions2 

50  ‘  EXPERIMENT  DATA  SET  5  -  FACES 
CLS  0 

set5f ace info 
set5faces 
WRITE  #1,  ’’SET  5” 
quest ionsl 

60  '  EXPERIMENT  DATA  SET  6  -  TABLES 

CLS  0 
chart3 

WRITE  #1,  "SET  6" 
quest ionsl 

70  '  EXPERIMENT  DATA  SET  7  -  FACES 
CLS  0 

set7faceinfo 
set7faces 
WRITE  #1.  "SET  7" 
quest ions2 

80  '  EXPERIMENT  DATA  SET  8  -  TABLES 

CLS  0 
chart4 

WRITE  #1,  "SET  8" 
quest ions2 

90  '  NOTICE  OF  EXPERIMENT  COMPLETION 
CLS  0 

VIEW  PRINT  1  TO  24 
LOCATE  4,  1 

PRINT  "That  completes  this  portion  of  the  experiment . " 

LOCATE  6.  1 

PRINT  "Please  raise  your  hand  to  indicate  that  you  have  finished." 
LOCATE  8,  1 

PRINT  "Disobey  the  next  command  by  not  pressing  any  key  to  continue." 
99  END 
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SUBROUTINES 


NOTE:  Because  several  subroutines  are  similar,  some  are  omitted  in  this 
test.  Those  subroutines  which  are  omitted  will  be  identified. 

SUB  chart  '  Lists  Tabular  Data  on  Output  Screen 

CLS 

VIEW  PRINT  1  TO  11 
LOCATE  1,  5 

PRINT  "  SCHEDULE 

GOST" 

LOCATE  2,  5 

PRINT  "ELEMENT  BCWS  BCWP  ACWP  VARIANCE 

VARIANCE" 

LOCATE  4,  5 

PRINT  USING  "  A  ######  ######  *#*#*#  #****# 

##*###";  bcws(5);  bcwp(5);  acwp(5) ;  sv(5);  cv(5) 

LOCATE  5,  5 

ppTKTT  UqirJf:  "  R  B  u  u  u  ini  H  ii  M  it  ^  u  u  y  h  u  limn 

I  liej  i-J  ■>  ii  ii  IV  IT  TT  II  II  II  n  n  n  II  Tl  TT  TT  II  H  TT  TT  TT  TT  TT  TT 

♦#####";  bcws(6);  bcwp(6);  acwp(6) ;  sv(6) ;  cv(6) 

LOCATE  6.  5 

PRINT  USING  "  C  ##*###  ######  ######  #**#*# 

######" ;  bcws(7);  bcwp(7) ;  acwp(7);  sv(7) ;  cv(7) 

LOCATE  8,  5 

PRTNT  (RTNG  "  TTTTAT  -BBB  w  BB  B-ttBtttttt  -BBH  H  BHBHHB 

UiJXI'W  1 V-7  1  r\  1  ,  it  ft  ft  ft  tt  ft  If  IT  ft  It  ft  H  1?  TT  ft  ft  ft  ft  fflrff  tt  It  tr 

lillHW;  bcws(8);  bcwp(8);  acwp(8);  sv(8) ;  cv(8) 

END  SUB 

SUB  chart 2  'OMITTED  -  SIMILAR  TO  "CHART"  FOR  a  -  13  THROUGH  16 

SUB  chart3  ’OMITTED  -  SIMILAR  TO  "CHART"  FOR  a  =  21  THROUGH  24 

SUB  chart4  'OMITTED  -  SIMILAR  TO  "CHART"  FOR  a  =  29  THROUGH  32 


SUB  cprinfo  'CALCULATES  CV,  SV,  CPI,  SPI,  AND  TCPI 

6000 
a  =  1 
6100 

IF  a  >  35  THEM  GOTO  6200 

cv(a)  -  bcwp(a)  -  acwp(a)  'cost  variance 

sv(a)  =  bcwp(a)  -  bcws(a)  'schedule  variance 

cvp(a)  -  cv(a)  /  bcwp(a)  'cost  variance  percentage 

svp(a)  -  sv(a)  /  bcws(a)  'schedule  variance  percentage 

i 

'  Performance  Indices 
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cpi(a)  -  bcwp(a)  /  acwp(a) 
spi(a)  «  bcwp(a)  /  bcws(a) 

i 

pc(a)  =  bcwp(a)  /  bac(a) 

ps(a)  =  acwp(a)  /  bac{a) 

} 

tcpi(a)  -  (bac(a)  -  bcwp(a)) 
a  =  a  +  1 
GOTO  6100 


'Cost  Performance  Index 
'Schedule  Performance  Index 

'Percent  complete 
‘percent  spent 

./  (bac(a)  -  acwp(a)) 


6200 
END  SUB 


SUB  Facelnfo  'CONVERTS  CPR  DATA  TO  FACE  INPUTS 


r  =  25 
mr  *  10 
er  =  7 
ear  =  3 
pr  =  .75 
era  -  .5 
eara  =  .8 


'face  radius 
'mouth  radius 
'eye  radius 
'ear  radius 
'pupil  radius 
'eye  aspect  ratio 
'ear  aspect  ratio 


a  -  1 

i 

Input  CPR  Data 


100 

IF  a  >  4  GOTO  190 


bc(a)  =  tcpi(a)  *  -.1  +  1 
IF  bc(a)  >  1  THEN  bc(a)  -  1 
IF  be (a)  <  .7  THEN  bc(a)  -  .7 

i 

me (a)  *  cpi(a)  -  1 

IF  me (a)  <  -1  THEN  me (a)  -  -1 

IF  mc(a)  >  1  THEN  mc(a)  -  1 

i 

nl  (a)  -  5  +  spi  (a)  *  5 

IF  nl (a)  >  15  THEN  nl(a)  -  15 

i 

ef(a)  »  cvp(a)  *  7 

IF  ef (a)  <  -7  THEN  ef(a)  =  -7 

IF  ef (a)  >  7  THEN  ef(a)  -  7 

i 

pf(a)  “  svp(a)  *  10 
IF  pf (a)  <  -5  THEN  pf(a)  -  -5 
IF  pf (a)  >  5  THEN  pf(a)  -  5 
a  ■  a  +  1 
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GOTO  luu 
190 

EM)  SUB 


SUB  Faces  'DRAWS  FACES 

i 

SCREEN  9 

VIEW  (15,  5)-(615,  125),  8,  4 
WINDOW  (0,  0)-(400,  100) 

a  •  1 

'  Draw  Face  Outline 


1000 

IF  a  >  4  GOTO  1290 


pi  =  3.14 


xface  «=  (a  *  100)  -  50 

CIRCLE  (xface,  50),  r,  ,  0,  pi,  1  ‘top  half 
CIRCLE  (xface,  50),  r,  ,  pi,  0,  bc(a) 'bottom  half 

i 

'  Draw  mouth  curvature 

i 

IF  me  (a)  <=  0  GOTO  1100  ELSE  GOTO  1150 

i 

1100  mc(a)  =  -mc(a) 

CIRCLE  (xface,  35),  mr,  ,  .2  *  pi,  .8  *  pi.  mc(a) 

GOTO  1200 
1150 

CIRCLE  (xface,  40),  mr,  ,  1  *  pi,  0  *  pi,  mc(a) 
1200 

'  eyes 

l 

CIRCLE  (xface  -  10,  60),  er,  .  ,  ,  era' left  eye 
CIRCLE  (xface  +  10,  60),  er,  ,  ,  ,  era'right  eye 

i 

1  nose 

i 

yn(a)  -  60  -  nl (a) 

LINE  (xface,  60)-(xface  +  3,  yn(a)) 

LINE  (xface  +  3.  yn(a) )-(xface  -  3,  yn(a)) 

LINE  (xface  -  3,  yn(a) ) -(xface,  60) 

i 

'  ears 

CIRCLE  (xface  +  29,  50),  ear.  .  ,  ,  eara 
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✓ 


CIRCLE  (xface  -  29,  50),  ear.  ,  ,  ,  eara 

t 

'  eyebrows 

i 

IF  ef  (a)  >  0  THEN  GOTO  1225 
yel(a)  -  65  +  ef(a) 

LINE  (xface  -  5,  65)-(xface  -  17,  yel(a)) 
LINE  (xface  +  5,  65)-(xface  +  17,  yel(a)) 
GOTO  1250 
1225 

yel (a)  -  65  -  ef (a) 

LINE  (xface  -  17.  65)-(xface  -  5,  yel (a)) 
LINE  (xface  +  17,  65)-(xface  +  5,  yel (a)) 
1250 


1  pupils 

t 

xl(a)  =  (xface  -  10)  +  pf(a) 
x2(a)  =  (xface  +  10)  +  pf(a) 

CIRCLE  (xl(a),  60),  pr 
PAINT  (xl (a) ,  60) 

CIRCLE  (x2(a),  60),  pr 
PAINT  (x2(a) ,  60) 
a  *=  a  +  1 
GOTO  1000 
1290 

VIEW  PRINT  10  TO  24 
LOCATE  11,  10 

PRINT  "A  B  C  OVERALL" 

END  SUB 


SUB  I  chart  ‘  LISTS  TABULAR  DATA  FOR  SAMPLE  FACES 

VIEW  PRINT  12  TO  24 
LOCATE  14,  10 

PRINT  "Cost  Performance  Information" 

LOCATE  16,  10 


PRINT  USING  "BCWS 
bcws(35) 

##### 

||  y  H 

n  n  h  fl  n 

#####" ;  bcws (33) ;  bcws(34) 

LOCATE  17.  10 

PRINT  USING  "BCWP 

uuuuu 

It  fl  fl  Trir 

MJJ  JLMH 

IMTIfnn 

#####";  bcwp(33) ;  bcwp(34) 

bcwp(35) 

LOCATE  18,  10 

PRINT  USING  "ACWP 

M  M  M  MM 
^  fl  If 

##### 

#####";  acwp(33);  acwp(34) 

acwp(35) 

LOCATE  19,  10 

PRINT  USING  "BAC 

##### 

UUUU1I 

ff  fl  W  W  fl 

#####" ;  bac(33);  bac(34); 

bac(35) 
END  SUB 
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SUB  If  a  ‘OMITTED  -  SAME  AS  "Ifa"  IN  CPR  FACES  PROGRAM  (APPENDIX  A) 
SUB  Ifi  ‘OMITTED  -  SAME  AS  “Ifi“  IN  CPR  FACES  PROGRAM  (APPENDIX  A) 


SUB  intro  'INTRO  SCREEN 
CLS  0 

VIEW  PRINT  6  TO  24 
LOCATE  7,  25 

PRINT  "Cost  Performance  Report  Faces  Demo" 
LOCATE  8,  35 

PRINT  "by  Capt  Jeffrey  T<ach" 

LOCATE  9,  38 
PRINT  "AFIT/GSM" 

1 

LOCATE  21.  10 

PRINT  "Press  Ary  Key  to  Continue" 
hit$  *  INKEYS 

IF  hits  <>  ""  THEN  GOTO  2  ELSE  GOTO  1 
2 

CLS 


END  SUB 


SUB  personal  "COLLECTS  BACKGROUND  DATA  ON  PARTICIPANT 

VIEW  PRINT  1  TO  24 
CLS 
20000 

INPUT  "What  is  your  current  AFSC  (or  civilian  equivalent)";  afscS 
20010 

PRINT  "You  answered".  afsc$ 

PRINT  "Is  this  correct  (Y/N)?“ 

DO 

rsp$  -  INKEYS 

LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  -  "N"  OR  rsp$  -  "n"  THEN  GOTO  20000 

20020 

CLS 

INPUT  "How  many  years  of  experience  do  you  have  at  that  AFSC";  yearsS 
20030 


96 


PRINT  "You  answered",  yearsS 
PRINT  "Is  this  correct  (Y/N)?" 

DO 

rsp$  =  INKEYS 
LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  -  "N"  OR  rsp$  -  "n"  THEN  GOTO  20020 

20040 

CLS 

INPUT  "Have  you  used  C/SCSC  before  (if  yes,  just  list  how  long)";  scsc$ 
20050 

PRINT  "You  answered",  scsc$ 

PRINT  "Is  this  correct  (Y/N)?" 

DO 

rsp$  =  INKEYS 

LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  -  "N"  OR  rsp$  -  "n"  THEN  GOTO  20040 

20060 

CLS 

INPUT  "What  is  rank/grade";  ranks 
20070 

PRINT  "You  answered",  ranks 
PRINT  “Is  this  correct  (Y/N)?" 

DO 

rspS  =  INKEYS 

LOOP  UNTIL  rspS  <>  "" 

IF  rspS  =  "N"  OR  rspS  -  "n"  THEN  GCT0  20060 


WRITE  #1,  afscS 

WRITE  #1,  yearsS  'Write  data  to  file 

WRITE  #1,  scscS 
WRITE  *1,  rankS 

END  SUB 


SUB  quest ions 1  'Qjestion  Set  1 

VIEW  PRINT  11  TO  24 

WRITE  #1.  "Start  Time  TIMES 


97 


'  Question  1 

i 

500 

LOCATE  12,  1 
FOR  i%  -  0  TO  11 
LOCATE  12  +  i%,  1 
PRINT  SPC(70); 

NEXT  i% 

LOCATE  13,  1 

PRINT  "Is  the  contractor  BEHIND  (B) ,  AHEAD  (A) ,  or  RIGHT  ON  (R) 
schedule?" 

i 

510 

l 

DO 

ansA$  *  INKEYS 

LOOP  UNTIL  ansAS  <>  "" 

IF  ansA$  -  "A"  OR  ansA$  -  "a"  OR  ansA$  -  "B"  OR  ansA$  -  "b"  THEM  GOTO 
520 

IF  ansA$  =  "R"  OR  ansAS  -  "r"  THEN  GOTO  520 
PRINT  "Answer  A,  B,  or  R  " 

GOTO  510 


520 


'Echo  Participant’s  Answer 
PRINT  USING  "Your  Answer  is  \  \";  ansA$ 

'Give  Opportunity  to  Change  Answer 
PRINT  "Do  You  Wish  to  Change  Your  Answer?  (Y/N)" 


DO 

rspAS  -  INKEYS 

LOOP  UNTIL  rspAS  <>  "" 

IF  rspA$  -  "Y"  OR  rspAS  -  /"  THEN  GOTO  500 


'Write  answer  to  file  "aidata.dat" 


WRITE  #1.  "1  ansAS 
525 

LOCATE  12,  1 
FOR  i%  -  0  TO  11 
LOCATE  12  +  i%,  1 
PRINT  SPC(70) ; 

NEXT  l% 


Question  2 
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530 

LOCATE  13,  1 

l 

PRINT  "Is  the  contractor  OVE3?  (0) .  UNDER  (U)  or  RIGHT  AT  (R)  cost" 

i 

540 

l 

DO 

ansB$  »■  INKEY$ 

LOOP  UNTIL  ansB$  <>  "" 

IF  ansB$  -  "o"  OR  ansB$  -  "0"  OR  ansB$  -  "u"  OR  ansB$  -  "U"  THEN  GOTO 
550 

IF  ansB$  -  "R"  OR  ansB$  =  "r"  THEN  GOTO  550 

PRINT  "Answer  0,  U,  or  R  " 

GOTO  540 
• 

550 

PRINT  USING  "Your  Answer  Is  \  \";  ansB$ 

i 

PRINT  "Do  You  Wish  to  Change  Your  Answer?  (Y/N)“ 

DO 

rsp$  =  INKEY$ 

LOOP  UNTIL  rsp$  <  >  "" 

IF  rsp$  -  "Y"  OR  rsp$  =  "y"  THEN  GCTO  525 
WRITE  #1.  "2  -",  ansBS 

555 

i 


LOCATE  12.  1 
FOR  i%  =  0  TO  11 
LOCATE  12  +  i%.  1 
PRINT  SPC(70) ; 
NEXT  i% 


Question  3 


560 

LOCATE  13,  1 


PRINT 

PRINT 

PRINT 

PRINT 

PRINT 

PRINT 

PRINT 

PRINT 

PRINT 


"What  elements  are  over  cost:" 
"(A)  Element  A  only" 

"(B)  Element  B  only" 

"(C)  Element  C  only" 

"(D)  Elements  A  and  B  only" 

"(E)  Elements  B  and  C  only" 

"(F)  Elements  A  and  C  only" 

"(G)  Elements  A.  B,  and  C" 

"(H)  No  elements  are  over  cost" 


570 


DO 
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ansC$  -  INKEY$ 

LOOP  UNTIL  ansC$  <>  "" 

IF  ansC$  -  "A"  OR  ansC$  -  "a"  OR  ansC$  -  "B"  OR  ansC$  -  "b" 
580 

IF  ansC$  =  "C"  OR  ansC$  «  "c"  OR  ansC$  -  "D”  OR  ansC$  -  “d" 
580 

IF  ansC$  ■=  "E"  OR  ansC$  =  "e"  OR  ansC$  -  "F“  OR  ansC$  -  "f" 
580 

IF  ansC$  ■=  "G"  OR  ansC$  -  "g"  OR  ansC$  -  "H"  OR  ansC$  -  "h" 
580 

t 

PRINT  "Answer  A,  B,  C,  D,  E,  F,  G  or  H" 

GOTO  570 

i 

580 

PRINT  USING  "Your  Answer  is  \  \";  ansC$ 

i 

PRINT  “Do  You  Wish  to  Change  Your  Answer?  (Y/N) " 

DO 

rsp$  ~  INKEY$ 

LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  *  "Y"  OR  rsp$  -  "y"  THEN  GOTO  555 

WRITE  #1,  "3  ansC$ 

» 

585 

• 

LOCATE  12,  1 
FOR  i%  -  0  TO  11 
LOCATE  12  +  i%,  1 
PRINT  SPC(70) ; 

NEXT  i% 

l 

'  Question  4 

i 

590 

LOCATE  13,  1 

i 

PRINT  "What  elements  are  behind  schedule:" 

PRINT  "(A)  Element  A  only" 

PRINT  " (B)  Element  B  only" 

PRINT  "(C)  Element  C  only" 

PRINT  "(D)  Elements  A  and  B  only" 

PRINT  "(E)  Elements  B  and  C  only" 

PRINT  "(F)  Elements  A  and  C  only" 

PRINT  "(G)  Elements  A,  B,  and  C" 

PRINT  "(H)  No  elements  are  behind  schedule" 

i 

600 

i 

DO 

ansD$  -  INKEY$ 

LOOP  UNTIL  ansD$  <>  "" 
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IF  ansD$  -  "A"  OR  ansD$  -  "a"  OR  ansDS  -  "B"  OR  ansD$  -  "b" 
610 

IF  ansD$  =  "C"  OR  ansD$  =  "c"  OR  ansDS  -  "D"  OR  ansD$  -  "d" 
610 

IF  ansD$  =  “E"  OR  ansD$  =  "e"  OR  ansDS  -  "F"  OR  ansD$  =  "f“ 
610 

IF  ansD$  -  "G"  OR  ansD$  -  "g"  a?  ansDS  -  "H"  OR  ansD$  -  "h" 
610 

PRINT  "Answer  A,  B,  C,  D,  E,  F  or  G" 

GOTO  600 

i 

610 

PRINT  USING  "Your  Answer  is  \  \";  ansD$ 

PRINT  "Do  You  Wish  to  Change  Your  Answer?  (Y/N)" 

DO 

rsp$  -  INKEY$ 

LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  -  "Y"  OR  rsp$  -  "/"  THEN  GOTO  585 
WRITE  #1.  "4  ansD$ 

WRITE  #1.  "stop  time  »  ",  TIMES 

LOCATE  12.  1 
FOR  a%  -  0  TO  11 
LOCATE  12  +  i%,  1 
PRINT  SPC(70) ; 

NEXT  i% 

EM)  SUB 


SUB  quest ions2  'QUESTION  SET  2 


Question  1 


WRITE  #1,  "Start  Time  TIMES 
10000 

VIEW  PRINT  11  TO  24 

LOCATE  12.  1 
FOR  1%  -  0  TO  11 
LOCATE  12  +  i%,  1 
PRINT  SPC(70) ; 

NEXT  i% 

LOCATE  13,  1 

PRINT  "What  is  the  trend  in  schedule  variance?:" 
PRINT  "(A)  Getting  Better" 

PRINT  "(B)  Getting  Worse" 
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PRINT  "(C)  Staying  the  Same" 

PRINT  “(D)  No  Detectable  Trend" 

i 

10010 
DO 

ansl$  “  INKEY$ 

LOOP  UNTIL  ansl$  <>  "" 

IF  ansl$  »  "A"  OR  ansl$ 

10020 

IF  ansl$  =  "C"  OR  ansl$ 

10020 

PRINT  "Answer  A.  B,  C,  D" 

GOTO  10010 

i 

10020 

PRINT  USING  "Your  Answer  is  \  \";  ansl$ 

PRINT  "Do  You  Wish  to  Change  Your  Answer?  (Y/N)" 
DO 

rsp$  -  INKEY$ 

LOOP  UNTIL  rsp$  <>  "" 

IF  rsp$  “  "Y"  OR  rsp$  -  "y"  THEN  GOTO  10000 
WRITE  #1.  "1  ansl$ 

10030 

LOCATE  12.  1 
FOR  i%  -  0  TO  11 
IOCATE  12  +  i*.  1 
PRINT  SPC(70) ; 

NEXT  i% 

l 

'  Question  2 

i 

10040 

LOCATE  13,  1 

i 

PRINT  "What  is  the  trend  in  cost  variance?" 
PRINT  "(A)  Getting  Better" 

PRINT  "(B)  Getting  Worse" 

PRINT  "(C)  Staying  the  Same" 

PRINT  "(D)  No  Detectable  Trend" 


"a"  OR  ansj$  -  "B"  OR  ansJ$  -  "b‘ 
"c"  OR  ansj$  -  "D"  OR  ansJ$  -  ”d' 


PRINT  "Answer  A,  B,  C,  or  D" 


10050 

DO 

ansJ$  “  INKEY$ 

LOOP  UNTIL  ansJ$  <>  "" 

IF  ansJ$  *•  "A"  OR  ansJ$  - 
10060 

IF  ansJ$  =  "C"  OR  ansJ$  - 
10060 


-  "a"  OR  ansl$  -  "B"  OR  ansl$  -  "b" 

-  "c"  OR  ansl$  -  "D"  OR  ansl$  ”  "d" 


THEN  GOTO 
THEN  GOTO 


'  THEN  GOTO 
'  THEN  GOTO 
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GOTO  10050 


10060 

PRINT  USING  “Your  Answer  is  \  \";  ansJ$ 

PRINT  “Do  You  Wish  to  Change  Your  Answer?  (Y/N)“ 
DO 

rsp$  -  INKEYS 
LOOP  UNTIL  rsp$  <> 

IF  rsp$  =  "Y"  OR  rsp$  -  "y"  THEN  GOTO  10030 

WRITE  #1.  "2  ansJS 

WRITE  #1,  "Stop  Time  TIMES 

LOCATE  12,  1 
FOR  i%  -  0  TO  11 
LOCATE  12  +  ifc,  1 
PRINT  SPC(70); 

NEXT  i% 

DO  SUB 


SUB  set3f  ace  info  'OMITTED  -  SIMILAR  TO  "faceinfo"  FOR  a  -  9  through  12 

SUB  set 3f aces  'OMITTED  -  SIMILAR  TO  "face"  FOR  a  -  9  through  12 

SUB  set5faceinfo  'OMITTED  -  SIMILAR  TO  "faceinfo"  FOR  a  -  17  through  20 

SUB  set5faces  'OMITTED  -  SIMILAR  TO  "face"  FOR  a  *  17  through  20 

SUB  set7faceinfo  'OMITTED  -  SIMILAR  TO  "faceinfo"  FOR  a  -  25  through  28 

SUB  set7faces  'OMITTED  -  SIMILAR  TO  "face"  FOR  a  ’>  25  through  28 
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Appendix  C:  Ebcperiment  Data 


The  input  data  for  the  experiment  was  fabricate  to  represent 
the  performance  of  contracts.  Thirty-two  data  sets  were  prepared 
representing  eight  individual  contracts.  Each  contract  was  described  by 
a  contract  data  group  consisting  of  four  data  sets.  There  are  two  types 
of  contract  data  groups.  Type  1  presents  three  data  sets  representing 
individual  elements  of  the  contract  and  a  fourth  data  set  representing 
overall  performance.  This  type  was  designed  to  test  an  individual’s 
ability  to  analyze  contract  performance.  Type  2  presents  four  data  sets 
representing  a  specific  contract  period.  The  data  for  each  period  is 
cumulative  (encompasses  data  from  previous  periods) .  This  type  was 
designed  to  test  an  individual's  ability  to  detect  trends  in  the  data. 
Type  1  contract  data  groups  are  shown  in  Table  28.  Type  2  Dat  Groups 
are  shown  in  Table  29. 

The  eight  contract  data  groups  were  sorted  eight  different  ways,  so 
that  the  experiment  participants  would  not  all  see  the  data  groups  in 
the  same  order.  The  contract  data  groups  were  copied  onto  floppy  disks 
which  also  contained  the  experiment  program.  A  total  of  thirty-two 
floppy  disks  were  used  (each  ordering  of  contract  groups  appeared  on 
four  disks).  The  disks  were  randomly  distributed  to  the  participants. 


104 


9 


TABLE  28 

Contract  Data  Group  Type  1  Data 


CONTRACT  DATA  GROUP  1 

ELEMENT 

BCWS 

ELEMENT 

BCWP 

ACWP 

BAC 

A 

3000 

3700 

4400 

5000 

B 

1700 

1400 

1100 

4000 

C 

6000 

4900 

6000 

7000 

OVERALL 

10700 

10000 

11500 

16000 

CONTRACT  DATA  GROUP  2 

A 

2300 

3000 

3500 

6000 

B 

3200 

3000 

2800 

6000 

C 

2000 

3000 

2700 

6000 

OVERALL 

7500 

9000 

9000 

18000 

CONTRACT  DATA  GROUP  3 

A 

7500 

7300 

7600 

8000 

B 

7400 

7400 

5700 

8000 

C 

5500 

7000 

5300 

8000 

OVERALL 

20400 

21700 

18600 

24000 

CONTRACT  DATA  GROUP  4 

A 

3000 

3000 

3000 

5000 

B 

1600 

1600 

2400 

4000 

C 

5000 

5000 

4600 

7000 

D 

9600 

9600 

12000 

16000 
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TABLE  29 


CONTRACT  DATA  GROUP  TYPE  2  DATA 


CONTRACT  DATA  GROUP  A 

PERIOD 

BCWS 

ELEMENT 

BCWP 

ACWP 

BAC 

1 

4900 

3600 

4000 

20000 

2 

9000 

7400 

8000 

20000 

3 

12500 

11400 

11400 

20000 

4 

15500 

15600 

14600 

20000 

CONTRACT  DATA  GROUP  B 

1 

4000 

3600 

3300 

20000 

2 

8000 

7400 

6900 

20000 

3 

12000 

11000 

10400 

20000 

4 

16000 

15000 

14400 

20000 

CONTRACT  DATA  GROUP  C 

1 

4000 

4200 

4600 

20000 

2 

8000 

8400 

9400 

20000 

3 

12000 

12600 

14400 

20000 

4 

16000 

16800 

19600 

20000 

CONTRACT  DATA  GROUP  D 

1 

4500 

3800 

4200 

20000 

2 

8000 

8000 

8000 

20000 

3 

12000 

12300 

11000 

20000 

4 

14500 

16500 

16700 

20000 
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Appendix  D:  Test  Scenario 


Cost  Performance  Information 


KBS  2080  10800  «H 
BCHP  4000  4000  4000 
ACW  2000  7000  4000 
BflC  15000  15000  15000 


Press  Any  Key  to  Continue 
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Is  the  Contractor  AHEAD  (A) ,  BEHIND  (B) ,  or  RIGHT  ON  (R)  Schedule? 


Is  the  Contractor  OVER  (0) .  UNDER  (U) ,  or  RIGHT  AT  (R)  Cost? 


What  Elements  are  OVER  Cost? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A,  B.  and  C 

F.  No  elements  are  over  cost. 


What  Elements  are  BEHIND  Schedule? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A,  B,  and  C 

F.  No  elements  are  over  cost. 
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KLOBNT 

BCUS 

•CUP 

A0UT 

SQOULE 

UAIIANCE 

GOST 

UARIAMCK 

A 

3MB 

37M 

44M 

7M 

-7M 

B 

i?se 

i486 

11M 

-308 

3M 

G 

MM 

49M 

MM 

-11M 

-11M 

TOTAL 

1B7M 

10MB 

115M 

-7M 

-IMS 

Is  the  Contractor  AHEAD  (A) .  BEHIND  (B) ,  or  RIGHT  ON  (R)  Schedule? 


Is  the  Contractor  OVER  (0) ,  UNDER  (U) ,  or  RIGHT  AT  (R)  Cost? 


What  Elements  are  OVER  Cost? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A,  B.  and  C 

F.  No  elements  are  over  cost. 


What  Elements  are  BEHIND  Schedule? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A.  B,  and  C 

F.  No  elements  are  over  cost. 
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i  iii 


What  is  the  trend  in  Schedule  Variance? 


A.  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 
B .  No  Trend 


What  is  the  trend  in  Cost  Variance? 

A.  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 

D .  No  Trend 
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<  CQ  O  Ci 


Is  the  Contractor  AHEAD  (A) ,  BEHIND  (B) ,  or  RIGHT  ON  (R)  Schedule? 


Is  the  Contractor  OVER  (0) ,  UNDER  (U) ,  or  RIGHT  AT  (R)  Cost? 


What  Elements  are  OVER  Cost? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A.  B,  and  C 

F.  No  elements  are  over  cost. 


What  Elements  are  BEHIND  Schedule? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A.  B,  and  C 

F.  No  elements  are  over  cost. 
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ELDBNT 


A 

B 

C 

TOTAL 


BCUS 


7588 


“  “  SCWMJLE  COST  | 

BMP  ACWP  VARIANCE 


8888  9888  1580  8 


Is  the  Contractor  AHEAD  (A) ,  BEHIND  (B) ,  or  RIGHT  ON  (R)  Schedule? 


Is  the  Contractor  OVER  (0) ,  UNDER  (U) ,  or  RIGHT  AT  (R)  Cost? 


What  Elements  are  OVER  Cost? 

A.  A  only 

B.  B  only 

C.  C  only 

D.  A  and  B  only 

E.  A  and  C  only 

F.  B  and  C  only 

G.  A,  B,  and  C 

F.  No  elements  are  over  cost. 


What  Elements  are  BEHIND  Schedule? 


A. 

A  only 

B. 

B  only 

C. 

C  only 

D. 

A  and  B  only 

*  E. 

A  and  C  only 

F. 

B  and  C  only 

G. 

A,  B,  and  C 

*  F. 

No  elements  are  over  cost. 
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What  is  the  trend  in  Schedule  Variance? 

A.  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 

D.  No  Trend 


What  is  the  trend  in  Cost  Variance? 

A.  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 

D .  No  Trend 


August 
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HUSKY 

BCUS 

ku 

ten 

8CW9ULE 

UM1ANCK 

COST 

UftlMMX 

Qtr  1 

4988 

3600 

4000 

-1300 

HR 

<ttr  2 

MB 

7400 

8000 

-1600 

-600 

(Hr  3 

12508 

11400 

11400 

-1100 

0 

(Hr  4 

1SG88 

15600 

14600 

100 

1000 

What  is  the  trend  in  Schedule  Variance? 

A  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 

D.  No  Trend 


What  is  the  trend  in  Cost  Variance? 

A.  Getting  Better 

B.  Getting  Worse 

C.  Staying  The  Same 

D.  No  Trend 
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